-- Logs begin at Thu 2022-07-28 02:35:01 UTC, end at Thu 2022-07-28 04:21:45 UTC. -- Jul 28 03:45:17.179524 dv-cinder-799871-47-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 03:45:17.187414 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 03:45:17.189395 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 03:45:17 2022] *** Jul 28 03:45:17.189395 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 03:45:17.189395 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 03:45:17.189395 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: nodename: dv-cinder-799871-47-iscsi Jul 28 03:45:17.189395 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: machine: x86_64 Jul 28 03:45:17.189395 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: clock source: unix Jul 28 03:45:17.189395 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: pcre jit disabled Jul 28 03:45:17.189830 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: detected number of CPU cores: 4 Jul 28 03:45:17.189830 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: current working directory: / Jul 28 03:45:17.189830 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: detected binary path: /usr/bin/uwsgi-core Jul 28 03:45:17.189830 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: your processes number limit is 47680 Jul 28 03:45:17.189830 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: your memory page size is 4096 bytes Jul 28 03:45:17.189830 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: detected max file descriptor number: 2048 Jul 28 03:45:17.189830 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: lock engine: pthread robust mutexes Jul 28 03:45:17.189830 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: thunder lock: enabled Jul 28 03:45:17.189830 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 03:45:17.189830 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 03:45:17.229865 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: Python main interpreter initialized at 0x56457f4a99c0 Jul 28 03:45:17.229865 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: python threads support enabled Jul 28 03:45:17.229865 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: your server socket listen backlog is limited to 100 connections Jul 28 03:45:17.229865 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: your mercy for graceful operations on workers is 90 seconds Jul 28 03:45:17.230249 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 03:45:17.230328 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: *** Operational MODE: preforking *** Jul 28 03:45:17.230600 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: *** uWSGI is running in multiple interpreter mode *** Jul 28 03:45:17.230600 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: spawned uWSGI master process (pid: 80460) Jul 28 03:45:17.230841 dv-cinder-799871-47-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 03:45:17.231925 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: spawned uWSGI worker 1 (pid: 80472, cores: 1) Jul 28 03:45:17.231925 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: spawned uWSGI worker 2 (pid: 80473, cores: 1) Jul 28 03:45:17.231925 dv-cinder-799871-47-iscsi devstack@keystone.service[80460]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 03:45:22.698773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 03:45:22.698773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: from cryptography.utils import int_from_bytes Jul 28 03:45:22.699601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 03:45:22.699601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: from cryptography.utils import int_from_bytes Jul 28 03:45:22.699736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 03:45:22.699736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: from cryptography.utils import int_from_bytes Jul 28 03:45:22.699736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 03:45:22.699736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: from cryptography.utils import int_from_bytes Jul 28 03:45:22.749283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 03:45:22.749475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 03:45:22.749666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] command line args: [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 03:45:22.749906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 03:45:22.749987 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 03:45:22.750097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ================================================================================ {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 03:45:22.750179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 03:45:22.750295 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] command line args: [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 03:45:22.750372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] admin_token = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.750569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 03:45:22.750646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] config_dir = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.750748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ================================================================================ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 03:45:22.750943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.750943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] config_source = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.751147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] admin_token = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.751263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] debug = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.751389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config_dir = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.751499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.752047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.752047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config_source = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.752472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.752699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] debug = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.752918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.753131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.753342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.753472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.753609 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.754735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.754916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.755005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.755128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_config_append = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.755128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.755313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.755389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_dir = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.755507 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.755586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_file = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.755656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_config_append = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.755723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.755846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.755846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_dir = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.756945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.757132 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.757209 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.757323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.757403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.757478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.757595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.757674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.757821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.757976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.758139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.758212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.758327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.758399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.758520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.758594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.758715 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.758715 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.758918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.758999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] publish_errors = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759304 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] publish_errors = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.759950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] standard_threads = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] standard_threads = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760429 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760429 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] use_journal = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] use_json = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] use_stderr = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.760973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] use_syslog = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.761046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_journal = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.761123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.761193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_json = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.761193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_stderr = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.761315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.761383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_syslog = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.761450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.761526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 03:45:22.761601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.761671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.761739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.761812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.761887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.761967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762623 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.762942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.763012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.763080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.763153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.763223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.763387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.763463 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.763561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.763638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.763800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.763882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.764049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.764133 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.764297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.764373 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.764536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.764611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.764785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.764909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.765027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.765144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.765254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.765371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.765492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.765609 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.765727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.765838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.765959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.766070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.766190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.766302 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.766411 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.766519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.766634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.766757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.766892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.767002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.767112 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.767222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.767342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.767461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.767572 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.767682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.767792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.767908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.768019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.768140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.768251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.768361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.768477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.768587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.768697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.768814 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.768927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.769036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.769146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.769255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.769365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.769590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.769710 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.769821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.770128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.770351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.770464 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.770588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.770702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.770842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.770942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.771035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.771128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.771241 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.771337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.771482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.771574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.771683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.771777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.771897 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.772000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.772109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.772219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.772328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.772436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.772554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.772671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.772785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.772894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.773004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.773137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.773259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.773894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.774048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.774163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.774275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.774385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.774497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.774607 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.774725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.774848 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.774964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.775092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.775207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.775325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.775436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.775555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.775668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.775781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.775898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.776017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.776130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.776242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.776362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.776483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.776595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.776714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.776847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.776955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.777066 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.777182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.777295 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.777414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.777526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.777636 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.777746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.777857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.778019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.778145 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.778388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.778504 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.778616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.778726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.778896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.779030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.779144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.779254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.779366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.779483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.779594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.779713 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.779831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.779949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.780060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.780180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.780293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.780413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.780525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.780635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.780746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.780856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.780975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.781095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.781207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.781317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.781429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.781540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.781650 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.781836 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.781951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.782061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.782172 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.782282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.782391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.782510 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.782631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.782754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.782895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.783015 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.783128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.783239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.783350 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.783469 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.783582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.783694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.783806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.783917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.784037 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.784150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.784262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.784373 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.784484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.784596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.784715 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.784828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.784946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.785058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.785169 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.785281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.external = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.785399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.785518 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.785631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.785742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.785862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.785975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.786093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.external = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.786233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.password = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.786394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.786525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.token = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.786927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.787045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.787158 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.787270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.787394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.password = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.787509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.787621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.token = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.787733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.787845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.787969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.788091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.788205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.788317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.788429 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.788541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.788653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.788773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.788887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.790063 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.790261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.790389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.790504 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.790627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.790742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.790872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.790986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.791098 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.791210 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.791335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.791456 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.791569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.791687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.791806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.791921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.792042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.792163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.792277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.792277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.792489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.792600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.792721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.792840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.792954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.793066 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.793178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.793289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.793409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.793409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.793621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.793621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.793831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.793831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.794051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.794165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.794276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.794389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.794507 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.794621 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.794742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.794901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.795015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.795128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.795240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.795352 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.795476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.795592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.795704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.795816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.795928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.796052 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.796167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.796279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.796391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.796503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.796614 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.796815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.796932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.797045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.797159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.797270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.797388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.797509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.797623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.797736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.797848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.797960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.798072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.798192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.798307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.798420 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.798532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.798644 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.798756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.798898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.799013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.799125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.799235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.799346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.799458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.799576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.799690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.799801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.799911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.800021 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.800132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.800251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.800366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.800551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.800668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.800781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.800893 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.801015 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.801128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.801241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.801354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.801466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.801577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.801698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.801811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.801924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.802035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.802147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.802374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.802505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.802620 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.802732 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.802895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.803010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.803123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.803248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.803363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.803476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.803589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.803701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.803813 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.803942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.804056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.804168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.804280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.804392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.804505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.804823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.804986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.805110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.805275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.805393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.805506 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.805630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.805744 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.805856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.805968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.806081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.806193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.806314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.806429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.806541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.806654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.806654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.806654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.806998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.807113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.807226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.807338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.807450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.807562 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.807685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.807799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.807911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.808023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.808135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.808255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.808369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.808482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.808701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.808822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.808935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.809056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.809170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.809283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.809395 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.809508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.809619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.809740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.809855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.809967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.810080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.810192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.810304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.810426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.810541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.810654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.810840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.810958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.811071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.811196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.811312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.811424 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.811537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.811649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.811761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.811882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.811996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.812109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.812221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.812333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.812446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.812570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.812570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.812783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.812783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.812994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.813107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.813227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.813346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.813460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.813572 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.813690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.813804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.813923 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.814038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.814150 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.814263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.814375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.814488 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.814607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.814721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.814857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.814974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.815086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.815198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.815318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.815431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.815543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.815654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.815766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.815877 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.815997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.816110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.816222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.816334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.816446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.816558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.816558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.816777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.816889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.817000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.817112 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.817223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.817343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.817456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.817567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.817679 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.817791 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.817902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.818022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.818136 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.818248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.818359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.818359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.818569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.818689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.818847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.818961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.818961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.819172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.819283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.819475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.819591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.819704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.819816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.819928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] role.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.820043 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.820164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] role.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.820292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.820409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.820568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.820714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.820828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.820950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.821064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.821178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.821292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.821405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.821525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.821639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.821752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.821864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.821976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.822089 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.822209 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.822324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.822324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.822535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.822647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.822759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.822901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.823016 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.823129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.823241 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.823353 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.823466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.823590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.823705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.823817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.823928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.824040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.824152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.824272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.824386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.824498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.824610 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.824721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.824834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.824954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.825067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.825178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.825290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.825402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.825514 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.825633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.825633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.825844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.825956 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.826075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.826195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.826309 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.826420 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.826533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.826533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.826742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.826742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.827011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.827123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.827235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.827346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.827457 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.827577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.827691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.827803 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.827914 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.828025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.828143 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.828264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.828378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.828488 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.828599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.828711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.828823 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.828942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.829055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.829166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.829277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.829388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.829499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.829618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.829733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.829847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.830047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.830167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.830279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.830400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.830513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.830624 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.830737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.830866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.830979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.831102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.831216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.831329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.831440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.831552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.831663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.831784 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.831898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.832009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.connection = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.832129 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.832252 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.832365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.832477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.832597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.832719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.832845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.832959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.833071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.833183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.833294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.833405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.833524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.833637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.833748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.833861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.833972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.834084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.834203 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.834318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.834429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.834541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.834652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.834763 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.834904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.835019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.835131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.835244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.835355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.835467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.835590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.835703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.835815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.836409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.836523 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.836636 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.836636 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 03:45:22.836856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.836856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.836856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 03:45:22.836856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 03:45:22.864761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.864990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.865217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.865493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.865725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.865948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.873477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.874299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.903354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.903576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.904099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.904336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.955850 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:22.956081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.006697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.006952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.007199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.007437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.071617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.071840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.072060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.072163 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.072277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.072360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.072491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.072573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.072705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.072797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.072969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.073186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.096019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.096134 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.096231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.096380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.096465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.096566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.096648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.096795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.096881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.097017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.097101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.097234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.097316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.097448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.108837 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.109012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.121462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.121708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.121838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.121921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.122038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.122122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.122266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.122357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.122494 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.122577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.122716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.122816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.122958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.123174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 03:45:23.139698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.139864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.139950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 03:45:23.140077 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 03:45:23.140224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.140412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.140495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 03:45:23.140615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 03:45:23.140698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.140886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.140970 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.141145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.141226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.141394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.141475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.141645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.151728 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.152000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.152131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.152263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.152406 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.152537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.152675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.152950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.159434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.159905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.164443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.164715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.164926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.165009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.165207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.165304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.165488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.165571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.165762 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.165842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.166023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.166106 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.166289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.166368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.166550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.166629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.166846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.166930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.167107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.167187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.167380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.167646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.190153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.190638 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.191659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.191946 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.192160 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.192437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.197130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.197402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.197605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.197687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.197887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.198167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.204318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.204596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.204847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.205131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.208723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.209016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.209271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.209353 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.209559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.209641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.209827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.209905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.210139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.210266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.210349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.210424 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.210623 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.210706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.210977 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.211255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.227094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.227393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.227924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.228213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.231319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.232178 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.234427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.234722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.235372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.235664 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.239108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.240070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.242397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.243501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.245650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.245942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.246221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.246511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.246669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.246816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.246995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.247107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.247278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.247569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.247854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.248137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.263461 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.263741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.264002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.264266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.264679 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.264963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.265228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.265491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.271120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.272397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.272648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.272920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.274584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.274884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.276458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.276735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.277026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.277300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.277577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.277722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.278001 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.278289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.278570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.278883 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.289369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.289648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.289922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.290195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.290471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.290559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.290749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.290860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.291052 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.291136 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.291393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.291670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.291939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.292211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.306723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.307823 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.309695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.310819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.312639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.313760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.315082 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.316230 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.316597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.316880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.317156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.317816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.318101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.318380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.324360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.325496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.325890 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.327126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.328821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.329107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.329386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.329665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.330051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.330338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.330616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.330924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.338336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.338618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.338920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.339669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.339954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.340242 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.345752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.346032 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.346306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.346579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.346872 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.347114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 03:45:23.347198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.347391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.347475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.347670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.347752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.347945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.348026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.348228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.348309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.348508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.348597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.348777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.348861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.349040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.349122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.349303 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.349411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.349693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.349973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.350243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.350516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.350775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 03:45:23.382389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56457f4a99c0 pid: 80473 (default app) Jul 28 03:45:23.384680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56457f4a99c0 pid: 80472 (default app) Jul 28 03:45:23.386186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effefd54-5efa-4fd6-be9a-fdd8259cbbb2 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:23.386477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effefd54-5efa-4fd6-be9a-fdd8259cbbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:23.386752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effefd54-5efa-4fd6-be9a-fdd8259cbbb2 None None] PATH_INFO: `/v3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:23.391036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Thu Jul 28 03:45:23 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:30.891099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c8b16-fdca-4f9d-ae57-7df63f6a91ac None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:30.891650 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c8b16-fdca-4f9d-ae57-7df63f6a91ac None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:30.891650 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c8b16-fdca-4f9d-ae57-7df63f6a91ac None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:30.896078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:45:30.901869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4833af-5980-4fad-b0fe-177991d38a0f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:30.902285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4833af-5980-4fad-b0fe-177991d38a0f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:30.902423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4833af-5980-4fad-b0fe-177991d38a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:30.972053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: INFO dbcounter [None req-0a4833af-5980-4fad-b0fe-177991d38a0f None None] Registered counter for database keystone Jul 28 03:45:30.992738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG oslo_db.sqlalchemy.engines [None req-0a4833af-5980-4fad-b0fe-177991d38a0f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80473) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 03:45:30.998983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writer thread running {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 03:45:31.176251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG passlib.handlers.bcrypt [None req-0a4833af-5980-4fad-b0fe-177991d38a0f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80473) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 03:45:31.176587 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG passlib.handlers.bcrypt [None req-0a4833af-5980-4fad-b0fe-177991d38a0f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80473) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 03:45:31.231170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0a4833af-5980-4fad-b0fe-177991d38a0f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:31.232769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0a4833af-5980-4fad-b0fe-177991d38a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:31.354542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 03:45:31.485915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c8b16-fdca-4f9d-ae57-7df63f6a91ac None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:31.556791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: INFO dbcounter [None req-757c8b16-fdca-4f9d-ae57-7df63f6a91ac None None] Registered counter for database keystone Jul 28 03:45:31.575808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG oslo_db.sqlalchemy.engines [None req-757c8b16-fdca-4f9d-ae57-7df63f6a91ac None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80472) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 03:45:31.581997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writer thread running {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 03:45:31.703846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c8b16-fdca-4f9d-ae57-7df63f6a91ac None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:31.708089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109b4b7-2ae9-4d15-a11b-26d31c4f9635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:31.708717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109b4b7-2ae9-4d15-a11b-26d31c4f9635 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:31.709010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109b4b7-2ae9-4d15-a11b-26d31c4f9635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:31.709285 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109b4b7-2ae9-4d15-a11b-26d31c4f9635 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:31.718075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109b4b7-2ae9-4d15-a11b-26d31c4f9635 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:31.769901 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3109b4b7-2ae9-4d15-a11b-26d31c4f9635 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 03:45:31.770196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109b4b7-2ae9-4d15-a11b-26d31c4f9635 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:31.780453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Thu Jul 28 03:45:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 03:45:31.786396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4833af-5980-4fad-b0fe-177991d38a0f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:31.801138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4833af-5980-4fad-b0fe-177991d38a0f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:31.806118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3078bc-7fa0-4126-acde-a7864075d8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:31.807248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3078bc-7fa0-4126-acde-a7864075d8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:31.807536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3078bc-7fa0-4126-acde-a7864075d8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:31.807813 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3078bc-7fa0-4126-acde-a7864075d8f2 admin admin] PATH_INFO: `/v3/projects/826fca2fab6740f6870c6426848e8d90` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:31.809295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3078bc-7fa0-4126-acde-a7864075d8f2 admin admin] RBAC: Authorizing `identity:get_project(project_id=826fca2fab6740f6870c6426848e8d90)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:31.861205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4e3078bc-7fa0-4126-acde-a7864075d8f2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 03:45:31.861547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3078bc-7fa0-4126-acde-a7864075d8f2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:31.866136 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Thu Jul 28 03:45:31 2022] GET /identity/v3/projects/826fca2fab6740f6870c6426848e8d90 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:33.836171 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a4c1f-90d5-434b-bf30-6905f9b1b234 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:33.836689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a4c1f-90d5-434b-bf30-6905f9b1b234 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:33.836787 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a4c1f-90d5-434b-bf30-6905f9b1b234 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:33.840914 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:33.846141 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40bf26-3834-47b6-93ff-1bfe2853c505 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:33.846550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40bf26-3834-47b6-93ff-1bfe2853c505 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:33.846709 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40bf26-3834-47b6-93ff-1bfe2853c505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:33.867777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bb40bf26-3834-47b6-93ff-1bfe2853c505 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:33.869371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bb40bf26-3834-47b6-93ff-1bfe2853c505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:33.877328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 03:45:33.899080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a4c1f-90d5-434b-bf30-6905f9b1b234 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:33.911063 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a4c1f-90d5-434b-bf30-6905f9b1b234 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:33.916395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6911c6-d4d7-43e5-9579-d0a79ffe2958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:33.917004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6911c6-d4d7-43e5-9579-d0a79ffe2958 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:33.917291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6911c6-d4d7-43e5-9579-d0a79ffe2958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:33.917572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6911c6-d4d7-43e5-9579-d0a79ffe2958 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:33.925286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6911c6-d4d7-43e5-9579-d0a79ffe2958 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:33.927090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6911c6-d4d7-43e5-9579-d0a79ffe2958 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:33.936331 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Thu Jul 28 03:45:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 03:45:33.941536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40bf26-3834-47b6-93ff-1bfe2853c505 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:33.950542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40bf26-3834-47b6-93ff-1bfe2853c505 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:33.954994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e731d-951b-4887-870c-44efa6ed66ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:33.955622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e731d-951b-4887-870c-44efa6ed66ae admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:33.955910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e731d-951b-4887-870c-44efa6ed66ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:33.956194 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e731d-951b-4887-870c-44efa6ed66ae admin admin] PATH_INFO: `/v3/users/fe2947ccbc264fdba860153b59471d41` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:33.957571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e731d-951b-4887-870c-44efa6ed66ae admin admin] RBAC: Authorizing `identity:get_user(user_id=fe2947ccbc264fdba860153b59471d41)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:33.959443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e731d-951b-4887-870c-44efa6ed66ae admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:33.961016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Thu Jul 28 03:45:33 2022] GET /identity/v3/users/fe2947ccbc264fdba860153b59471d41 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:36.004892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe28db0-b246-4919-868f-db8786d4f2bb None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:36.005426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe28db0-b246-4919-868f-db8786d4f2bb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:36.005426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe28db0-b246-4919-868f-db8786d4f2bb None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:36.006771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:36.012411 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee77c-2dc8-45f6-be5d-48191fa034d2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:36.012822 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee77c-2dc8-45f6-be5d-48191fa034d2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:36.012976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee77c-2dc8-45f6-be5d-48191fa034d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:36.027044 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2be55-f902-4cc5-b43c-57f09fc821c2 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:36.027341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2be55-f902-4cc5-b43c-57f09fc821c2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:36.027622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2be55-f902-4cc5-b43c-57f09fc821c2 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:36.028952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:45:36.033587 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a9bee77c-2dc8-45f6-be5d-48191fa034d2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:36.033821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b6d0a-1207-4057-b549-5deddf5be4b6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:36.034112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b6d0a-1207-4057-b549-5deddf5be4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:36.034394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b6d0a-1207-4057-b549-5deddf5be4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:36.034816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a9bee77c-2dc8-45f6-be5d-48191fa034d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:36.076389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG passlib.handlers.bcrypt [None req-b46b6d0a-1207-4057-b549-5deddf5be4b6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80472) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 03:45:36.076813 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG passlib.handlers.bcrypt [None req-b46b6d0a-1207-4057-b549-5deddf5be4b6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80472) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 03:45:36.116462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:36.122587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b46b6d0a-1207-4057-b549-5deddf5be4b6 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:36.124200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b46b6d0a-1207-4057-b549-5deddf5be4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:36.139015 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bee77c-2dc8-45f6-be5d-48191fa034d2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:36.163410 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:36.224907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:45:36.245715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bee77c-2dc8-45f6-be5d-48191fa034d2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:36.246032 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b6d0a-1207-4057-b549-5deddf5be4b6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:36.280913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b70e2-c842-49af-804e-a23dbdf3aae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:36.281500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b70e2-c842-49af-804e-a23dbdf3aae9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:36.281790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b70e2-c842-49af-804e-a23dbdf3aae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:36.282072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b70e2-c842-49af-804e-a23dbdf3aae9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:36.293549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b70e2-c842-49af-804e-a23dbdf3aae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1f9b70e2-c842-49af-804e-a23dbdf3aae9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:36.302344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:36.303293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:45:36.305048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:45:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:45:36.309860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b70e2-c842-49af-804e-a23dbdf3aae9 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:36.347727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b6d0a-1207-4057-b549-5deddf5be4b6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:36.349987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:36.385739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeaf0a-d515-4ba9-8381-99182b63a43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:36.386598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeaf0a-d515-4ba9-8381-99182b63a43c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:36.386906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeaf0a-d515-4ba9-8381-99182b63a43c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:36.387182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeaf0a-d515-4ba9-8381-99182b63a43c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:36.390936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:36.409437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b70e2-c842-49af-804e-a23dbdf3aae9 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:36.451357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0c1e2-dea7-47c9-bfad-4f5205f568b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:36.452090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0c1e2-dea7-47c9-bfad-4f5205f568b4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:36.452263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0c1e2-dea7-47c9-bfad-4f5205f568b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:36.452537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0c1e2-dea7-47c9-bfad-4f5205f568b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:36.453266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0c1e2-dea7-47c9-bfad-4f5205f568b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:36.453567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0c1e2-dea7-47c9-bfad-4f5205f568b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:36.454988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0c1e2-dea7-47c9-bfad-4f5205f568b4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:36.455187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceeaf0a-d515-4ba9-8381-99182b63a43c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:36.456699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceeaf0a-d515-4ba9-8381-99182b63a43c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:36.466860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:45:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:36.472181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0c1e2-dea7-47c9-bfad-4f5205f568b4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:36.558772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 8/16] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:45:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 03:45:36.564252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeaf0a-d515-4ba9-8381-99182b63a43c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:36.567006 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:36.572945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0c1e2-dea7-47c9-bfad-4f5205f568b4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:36.668376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeaf0a-d515-4ba9-8381-99182b63a43c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:36.704709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f410e-e571-42fb-a305-03e6682470cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:36.705564 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f410e-e571-42fb-a305-03e6682470cb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:36.705844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f410e-e571-42fb-a305-03e6682470cb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:36.706119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f410e-e571-42fb-a305-03e6682470cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:36.713084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d7eea-1d53-4f9b-a371-5502699437aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:36.713725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d7eea-1d53-4f9b-a371-5502699437aa None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:36.714009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d7eea-1d53-4f9b-a371-5502699437aa None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:36.714290 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d7eea-1d53-4f9b-a371-5502699437aa None admin] PATH_INFO: `/v3/users/fe2947ccbc264fdba860153b59471d41` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:36.715481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d7eea-1d53-4f9b-a371-5502699437aa None admin] RBAC: Authorizing `identity:get_user(user_id=fe2947ccbc264fdba860153b59471d41)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:36.716933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d7eea-1d53-4f9b-a371-5502699437aa None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:36.718525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 9/17] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:36 2022] GET /identity/v3/users/fe2947ccbc264fdba860153b59471d41 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:36.723893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d7eea-1d53-4f9b-a371-5502699437aa None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:36.724086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f410e-e571-42fb-a305-03e6682470cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:36.725532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f410e-e571-42fb-a305-03e6682470cb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:36.738954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-370f410e-e571-42fb-a305-03e6682470cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:45:36.744829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:36.745804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:36.746891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:45:36.746891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:45:36.746891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:45:36.746891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:45:36.746891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 9/18] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:36.751719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f410e-e571-42fb-a305-03e6682470cb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:36.808122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:36.826726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d7eea-1d53-4f9b-a371-5502699437aa None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:36.851728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f410e-e571-42fb-a305-03e6682470cb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:36.860090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f1114-1282-4d5c-98d5-fd9e2e9780f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:36.860737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f1114-1282-4d5c-98d5-fd9e2e9780f0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:36.861027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f1114-1282-4d5c-98d5-fd9e2e9780f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:36.861305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f1114-1282-4d5c-98d5-fd9e2e9780f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:36.862474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f1114-1282-4d5c-98d5-fd9e2e9780f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:36.864873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f1114-1282-4d5c-98d5-fd9e2e9780f0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:36.866449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 10/19] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:45:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:36.872679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f1114-1282-4d5c-98d5-fd9e2e9780f0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:36.886182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6b6cb-4b14-4155-bf14-d1f8142dd8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:36.886822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6b6cb-4b14-4155-bf14-d1f8142dd8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:36.887108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6b6cb-4b14-4155-bf14-d1f8142dd8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:36.887381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6b6cb-4b14-4155-bf14-d1f8142dd8d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:36.888073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6b6cb-4b14-4155-bf14-d1f8142dd8d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:36.888374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6b6cb-4b14-4155-bf14-d1f8142dd8d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:36.889754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6b6cb-4b14-4155-bf14-d1f8142dd8d3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:36.907382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 10/20] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:36.956298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:36.977885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f1114-1282-4d5c-98d5-fd9e2e9780f0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:37.013226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0f24d-fb3e-4859-b1fb-1de16f594131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:37.013907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0f24d-fb3e-4859-b1fb-1de16f594131 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:37.014142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0f24d-fb3e-4859-b1fb-1de16f594131 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:37.014422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0f24d-fb3e-4859-b1fb-1de16f594131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:37.015278 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0f24d-fb3e-4859-b1fb-1de16f594131 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.domain.id=default, user.id=fe2947ccbc264fdba860153b59471d41) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:37.015596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0f24d-fb3e-4859-b1fb-1de16f594131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:37.017076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0f24d-fb3e-4859-b1fb-1de16f594131 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:37.025659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 11/21] 10.222.0.46 () {56 vars in 1322 bytes} [Thu Jul 28 03:45:36 2022] GET /identity/v3/role_assignments?user.id=fe2947ccbc264fdba860153b59471d41&scope.domain.id=default&role.id=a37ae24c516b48c69e13721c013134a2 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:39.268522 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ad978-ab01-40cd-aa88-20e7975646a1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:39.269554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ad978-ab01-40cd-aa88-20e7975646a1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:39.270056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ad978-ab01-40cd-aa88-20e7975646a1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:39.271635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 11/22] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:39.287688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785bf71-212e-4d31-b98a-74c62a7a342d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:39.288160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785bf71-212e-4d31-b98a-74c62a7a342d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:39.288251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785bf71-212e-4d31-b98a-74c62a7a342d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:39.315801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f785bf71-212e-4d31-b98a-74c62a7a342d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:39.317010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f785bf71-212e-4d31-b98a-74c62a7a342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:39.403066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 12/23] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:39.424013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc844ac-2086-48c6-9e35-bfd3bb067f44 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:39.424606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc844ac-2086-48c6-9e35-bfd3bb067f44 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:39.425102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc844ac-2086-48c6-9e35-bfd3bb067f44 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:39.426666 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 12/24] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:39.432192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d107e-73d1-4f97-97d5-a0ccd3857426 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:39.432478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d107e-73d1-4f97-97d5-a0ccd3857426 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:39.432756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d107e-73d1-4f97-97d5-a0ccd3857426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:39.448897 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc844ac-2086-48c6-9e35-bfd3bb067f44 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:39.458929 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-835d107e-73d1-4f97-97d5-a0ccd3857426 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:39.460106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-835d107e-73d1-4f97-97d5-a0ccd3857426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:39.621470 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 13/25] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:39.660649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc844ac-2086-48c6-9e35-bfd3bb067f44 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:39.669188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d107e-73d1-4f97-97d5-a0ccd3857426 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:39.675929 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:39.691140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:39.713942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40ae0d-6361-4190-8c92-cc981d7248c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:39.714578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40ae0d-6361-4190-8c92-cc981d7248c2 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:39.714887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40ae0d-6361-4190-8c92-cc981d7248c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:39.715160 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40ae0d-6361-4190-8c92-cc981d7248c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:39.715896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40ae0d-6361-4190-8c92-cc981d7248c2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:39.717292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40ae0d-6361-4190-8c92-cc981d7248c2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:39.853492 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d107e-73d1-4f97-97d5-a0ccd3857426 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:39.892888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ee01f-4cfc-40cd-a9bd-6dff0d467b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:39.893817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ee01f-4cfc-40cd-a9bd-6dff0d467b88 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:39.894253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ee01f-4cfc-40cd-a9bd-6dff0d467b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:39.894681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ee01f-4cfc-40cd-a9bd-6dff0d467b88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:39.906340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ee01f-4cfc-40cd-a9bd-6dff0d467b88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:39.908119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ee01f-4cfc-40cd-a9bd-6dff0d467b88 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-7f3ee01f-4cfc-40cd-a9bd-6dff0d467b88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:45:39.925018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:39.925963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:39.926953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:45:39.926953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:45:39.926953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:45:39.926953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:45:39.928345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 14/26] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:45:39.929790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9cf1e-1a4d-4961-8da3-836fc1e97237 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:39.930240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9cf1e-1a4d-4961-8da3-836fc1e97237 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:39.930654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9cf1e-1a4d-4961-8da3-836fc1e97237 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:39.932040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 15/27] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:39.936970 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2ce07-6ba3-4b51-8cf2-7d39ed222348 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:39.937550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2ce07-6ba3-4b51-8cf2-7d39ed222348 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:39.937982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2ce07-6ba3-4b51-8cf2-7d39ed222348 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:39.939378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 16/28] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:39.940204 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2ce07-6ba3-4b51-8cf2-7d39ed222348 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:39.977662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 13/29] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Jul 28 03:45:39 2022] POST /identity/v3/roles => generated 219 bytes in 529 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:45:39.979165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25add9b1-e6e8-4c9a-a714-51bb84a9448e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:39.979456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25add9b1-e6e8-4c9a-a714-51bb84a9448e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:39.979736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25add9b1-e6e8-4c9a-a714-51bb84a9448e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:39.999087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-25add9b1-e6e8-4c9a-a714-51bb84a9448e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:40.000733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-25add9b1-e6e8-4c9a-a714-51bb84a9448e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:40.006186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:40.059591 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2ce07-6ba3-4b51-8cf2-7d39ed222348 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:40.078743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 14/30] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:40.080177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb99a2-2af5-4ff6-9718-731cb941871c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:40.080476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb99a2-2af5-4ff6-9718-731cb941871c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:40.080756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb99a2-2af5-4ff6-9718-731cb941871c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:40.099167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16148d6-36bb-4b57-9001-a506d3063456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:40.099590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b7eb99a2-2af5-4ff6-9718-731cb941871c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:40.100321 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16148d6-36bb-4b57-9001-a506d3063456 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:40.100776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16148d6-36bb-4b57-9001-a506d3063456 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:40.100875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b7eb99a2-2af5-4ff6-9718-731cb941871c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:40.101502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16148d6-36bb-4b57-9001-a506d3063456 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:40.102577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16148d6-36bb-4b57-9001-a506d3063456 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:40.103183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16148d6-36bb-4b57-9001-a506d3063456 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:40.104948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16148d6-36bb-4b57-9001-a506d3063456 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:40.125473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 17/31] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:40.126613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16148d6-36bb-4b57-9001-a506d3063456 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:40.177474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 15/32] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:40.178317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb99a2-2af5-4ff6-9718-731cb941871c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:40.235378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:40.253353 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16148d6-36bb-4b57-9001-a506d3063456 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:40.272408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:40.294423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5f8b9-a763-4a97-bcc1-eec1493cf262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:40.295888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5f8b9-a763-4a97-bcc1-eec1493cf262 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:40.296347 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5f8b9-a763-4a97-bcc1-eec1493cf262 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:40.296794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5f8b9-a763-4a97-bcc1-eec1493cf262 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:40.308640 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5f8b9-a763-4a97-bcc1-eec1493cf262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:40.314401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb99a2-2af5-4ff6-9718-731cb941871c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-aae5f8b9-a763-4a97-bcc1-eec1493cf262 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:40.315193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:40.315998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:45:40.316746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:45:40.319188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 18/33] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:45:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:40.320223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5f8b9-a763-4a97-bcc1-eec1493cf262 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:40.353499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82468769-8dfe-47dd-8e18-f21e474da3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:40.354825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82468769-8dfe-47dd-8e18-f21e474da3f7 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:40.354825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82468769-8dfe-47dd-8e18-f21e474da3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:40.354825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82468769-8dfe-47dd-8e18-f21e474da3f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:40.355766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82468769-8dfe-47dd-8e18-f21e474da3f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:40.357465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82468769-8dfe-47dd-8e18-f21e474da3f7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:40.393953 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 16/34] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:45:40 2022] POST /identity/v3/projects => generated 301 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:45:40.394971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82468769-8dfe-47dd-8e18-f21e474da3f7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:40.439798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5f8b9-a763-4a97-bcc1-eec1493cf262 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:40.480868 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2a8a5-5cf0-44ef-8523-2b76814b70e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:40.481866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2a8a5-5cf0-44ef-8523-2b76814b70e6 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:40.482476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2a8a5-5cf0-44ef-8523-2b76814b70e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:40.482986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2a8a5-5cf0-44ef-8523-2b76814b70e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:40.483931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2a8a5-5cf0-44ef-8523-2b76814b70e6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:40.485580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2a8a5-5cf0-44ef-8523-2b76814b70e6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:40.603993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:40.613314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82468769-8dfe-47dd-8e18-f21e474da3f7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:40.652751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eee46-b9d0-4142-9f6f-0abc6c3eed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:40.654569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eee46-b9d0-4142-9f6f-0abc6c3eed46 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:40.654569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eee46-b9d0-4142-9f6f-0abc6c3eed46 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:40.654569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eee46-b9d0-4142-9f6f-0abc6c3eed46 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:40.654843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23eee46-b9d0-4142-9f6f-0abc6c3eed46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:40.655000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23eee46-b9d0-4142-9f6f-0abc6c3eed46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:40.656684 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23eee46-b9d0-4142-9f6f-0abc6c3eed46 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:40.661077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 19/35] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Jul 28 03:45:40 2022] POST /identity/v3/roles => generated 225 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:45:40.671849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 17/36] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:45:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:40.678536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2a8a5-5cf0-44ef-8523-2b76814b70e6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:40.742213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:40.834768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2a8a5-5cf0-44ef-8523-2b76814b70e6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:40.872935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d91dd-d622-4164-84e9-8b1bd80bdc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:40.873519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d91dd-d622-4164-84e9-8b1bd80bdc02 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:40.873810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d91dd-d622-4164-84e9-8b1bd80bdc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:40.874093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d91dd-d622-4164-84e9-8b1bd80bdc02 None admin] PATH_INFO: `/v3/users/fe2947ccbc264fdba860153b59471d41` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:40.875491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d91dd-d622-4164-84e9-8b1bd80bdc02 None admin] RBAC: Authorizing `identity:get_user(user_id=fe2947ccbc264fdba860153b59471d41)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:40.877086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d91dd-d622-4164-84e9-8b1bd80bdc02 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:40.879176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 20/37] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:40 2022] GET /identity/v3/users/fe2947ccbc264fdba860153b59471d41 => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:40.884204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eee46-b9d0-4142-9f6f-0abc6c3eed46 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:40.931350 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39facc66-22ca-4a77-acec-0b8e5fe8540b None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:40.931637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39facc66-22ca-4a77-acec-0b8e5fe8540b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:40.931919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39facc66-22ca-4a77-acec-0b8e5fe8540b None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:40.933260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 21/38] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:45:40.934277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab33e70-f321-4371-9cdf-1af438e74d53 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:40.934624 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab33e70-f321-4371-9cdf-1af438e74d53 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:40.934936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab33e70-f321-4371-9cdf-1af438e74d53 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:40.936124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 22/39] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:40.938691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e559-7f1a-4d6e-a47d-ada4ed64c086 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:40.939022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e559-7f1a-4d6e-a47d-ada4ed64c086 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:40.939310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e559-7f1a-4d6e-a47d-ada4ed64c086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:40.958221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1048e559-7f1a-4d6e-a47d-ada4ed64c086 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:40.959794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1048e559-7f1a-4d6e-a47d-ada4ed64c086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:40.996887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eee46-b9d0-4142-9f6f-0abc6c3eed46 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:41.025327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 23/40] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:45:41.026359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6828b-1b54-4573-ac59-0cb4717c568d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:41.026650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6828b-1b54-4573-ac59-0cb4717c568d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:41.027075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6828b-1b54-4573-ac59-0cb4717c568d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:41.032658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a7f03-7100-47a1-84b5-1ca02a2afa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:41.033307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a7f03-7100-47a1-84b5-1ca02a2afa7b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:41.033597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a7f03-7100-47a1-84b5-1ca02a2afa7b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:41.033878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a7f03-7100-47a1-84b5-1ca02a2afa7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:41.035065 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a7f03-7100-47a1-84b5-1ca02a2afa7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:41.036514 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a7f03-7100-47a1-84b5-1ca02a2afa7b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:41.038563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 18/41] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:45:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:41.043929 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a7f03-7100-47a1-84b5-1ca02a2afa7b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:41.046766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f4c6828b-1b54-4573-ac59-0cb4717c568d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:41.047978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f4c6828b-1b54-4573-ac59-0cb4717c568d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:41.063106 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:41.076641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:41.117470 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 24/42] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:41.118365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6828b-1b54-4573-ac59-0cb4717c568d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:41.149189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a7f03-7100-47a1-84b5-1ca02a2afa7b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:41.183701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea794534-7d9b-4bec-88e1-e4c9041c247c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:41.184379 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea794534-7d9b-4bec-88e1-e4c9041c247c None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:41.184667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea794534-7d9b-4bec-88e1-e4c9041c247c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:41.184958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea794534-7d9b-4bec-88e1-e4c9041c247c None admin] PATH_INFO: `/v3/domains/default/users/fe2947ccbc264fdba860153b59471d41/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:41.187228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea794534-7d9b-4bec-88e1-e4c9041c247c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fe2947ccbc264fdba860153b59471d41, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:41.188685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea794534-7d9b-4bec-88e1-e4c9041c247c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:41.204571 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 19/43] 10.222.0.46 () {58 vars in 1361 bytes} [Thu Jul 28 03:45:41 2022] PUT /identity/v3/domains/default/users/fe2947ccbc264fdba860153b59471d41/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 03:45:41.205615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea794534-7d9b-4bec-88e1-e4c9041c247c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:41.221969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6828b-1b54-4573-ac59-0cb4717c568d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:41.261462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c29bb-7573-4246-b209-e41d7528f257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:41.262115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c29bb-7573-4246-b209-e41d7528f257 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:41.262400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c29bb-7573-4246-b209-e41d7528f257 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:41.262689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c29bb-7573-4246-b209-e41d7528f257 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:41.263503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c29bb-7573-4246-b209-e41d7528f257 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:41.264946 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c29bb-7573-4246-b209-e41d7528f257 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:41.281598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 25/44] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Jul 28 03:45:41 2022] POST /identity/v3/roles => generated 223 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:45:41.301175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:41.329757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea794534-7d9b-4bec-88e1-e4c9041c247c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:41.374384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f0e7a-7acd-4cc8-9729-cba6b0129b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:41.375119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f0e7a-7acd-4cc8-9729-cba6b0129b80 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:41.375393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f0e7a-7acd-4cc8-9729-cba6b0129b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:41.375676 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f0e7a-7acd-4cc8-9729-cba6b0129b80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:41.377012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f0e7a-7acd-4cc8-9729-cba6b0129b80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:41.378525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f0e7a-7acd-4cc8-9729-cba6b0129b80 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:41.380445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 20/45] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:45:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:41.386725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c29bb-7573-4246-b209-e41d7528f257 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:41.417135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:41.504720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c29bb-7573-4246-b209-e41d7528f257 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:41.541247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc55c8c-cc73-434b-9e8c-977ae2354128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:41.541869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc55c8c-cc73-434b-9e8c-977ae2354128 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:41.542151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc55c8c-cc73-434b-9e8c-977ae2354128 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:41.542427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc55c8c-cc73-434b-9e8c-977ae2354128 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:41.543146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc55c8c-cc73-434b-9e8c-977ae2354128 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:41.544645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc55c8c-cc73-434b-9e8c-977ae2354128 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:41.577442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 26/46] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:45:41 2022] POST /identity/v3/projects => generated 312 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:45:43.216954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1324d2-3fbf-4ee5-b38d-1ecc0dd9712e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:43.217469 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1324d2-3fbf-4ee5-b38d-1ecc0dd9712e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:43.217549 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1324d2-3fbf-4ee5-b38d-1ecc0dd9712e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:43.218932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 21/47] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:43.224187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823074cb-3138-43e2-bdca-f7d66d9d62d7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:43.224588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823074cb-3138-43e2-bdca-f7d66d9d62d7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:43.224748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823074cb-3138-43e2-bdca-f7d66d9d62d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:43.243143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-823074cb-3138-43e2-bdca-f7d66d9d62d7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:43.244328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-823074cb-3138-43e2-bdca-f7d66d9d62d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:43.309680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 27/48] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:43.332266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1324d2-3fbf-4ee5-b38d-1ecc0dd9712e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:43.433572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1324d2-3fbf-4ee5-b38d-1ecc0dd9712e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:43.466837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f912aea-5537-44d6-a6da-725c23c5bbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:43.467528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f912aea-5537-44d6-a6da-725c23c5bbde None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:43.467757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f912aea-5537-44d6-a6da-725c23c5bbde None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:43.468028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f912aea-5537-44d6-a6da-725c23c5bbde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:43.479512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f912aea-5537-44d6-a6da-725c23c5bbde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2f912aea-5537-44d6-a6da-725c23c5bbde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:43.485511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:43.486410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:45:43.487263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 22/49] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:45:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:43.492616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823074cb-3138-43e2-bdca-f7d66d9d62d7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:43.500832 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:43.555386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e9a18-2ae7-4624-8ce3-37a684605eaa None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:43.555680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e9a18-2ae7-4624-8ce3-37a684605eaa None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:43.555968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e9a18-2ae7-4624-8ce3-37a684605eaa None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:43.557322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 23/50] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:45:43.562161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f14e36-c843-45b8-aca7-c41da27959ca None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:43.562458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f14e36-c843-45b8-aca7-c41da27959ca None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:43.562926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f14e36-c843-45b8-aca7-c41da27959ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:43.577155 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:43.582941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a6f14e36-c843-45b8-aca7-c41da27959ca None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:43.584149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a6f14e36-c843-45b8-aca7-c41da27959ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:43.597368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823074cb-3138-43e2-bdca-f7d66d9d62d7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:43.631625 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a0316-b163-42fd-bcef-7350885e7dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:43.632798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a0316-b163-42fd-bcef-7350885e7dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:43.633051 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a0316-b163-42fd-bcef-7350885e7dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:43.633417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a0316-b163-42fd-bcef-7350885e7dd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:43.634154 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a0316-b163-42fd-bcef-7350885e7dd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:43.634461 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a0316-b163-42fd-bcef-7350885e7dd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:43.635880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a0316-b163-42fd-bcef-7350885e7dd1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:43.643958 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 28/51] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:45:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:43.650074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a0316-b163-42fd-bcef-7350885e7dd1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:43.656049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 24/52] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:45:43.677892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f14e36-c843-45b8-aca7-c41da27959ca None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:43.715144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:43.748957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a0316-b163-42fd-bcef-7350885e7dd1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:43.777425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f14e36-c843-45b8-aca7-c41da27959ca None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:43.782705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77cc32-c62a-4865-9e07-6d3052aef708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:43.783389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77cc32-c62a-4865-9e07-6d3052aef708 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:43.783680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77cc32-c62a-4865-9e07-6d3052aef708 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:43.783964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77cc32-c62a-4865-9e07-6d3052aef708 None admin] PATH_INFO: `/v3/users/fe2947ccbc264fdba860153b59471d41` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:43.785156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77cc32-c62a-4865-9e07-6d3052aef708 None admin] RBAC: Authorizing `identity:get_user(user_id=fe2947ccbc264fdba860153b59471d41)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:43.786915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77cc32-c62a-4865-9e07-6d3052aef708 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:43.788927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 29/53] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:43 2022] GET /identity/v3/users/fe2947ccbc264fdba860153b59471d41 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:43.794056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77cc32-c62a-4865-9e07-6d3052aef708 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:43.811873 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ad5c7-7a58-47e2-a6bb-5477af72071d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:43.812526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ad5c7-7a58-47e2-a6bb-5477af72071d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:43.812816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ad5c7-7a58-47e2-a6bb-5477af72071d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:43.813102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ad5c7-7a58-47e2-a6bb-5477af72071d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:43.814263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ad5c7-7a58-47e2-a6bb-5477af72071d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:43.815773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ad5c7-7a58-47e2-a6bb-5477af72071d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:43.817353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 25/54] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:45:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:43.823469 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ad5c7-7a58-47e2-a6bb-5477af72071d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:43.826934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:43.892945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77cc32-c62a-4865-9e07-6d3052aef708 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:43.924528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ad5c7-7a58-47e2-a6bb-5477af72071d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:43.927633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618c8e3-20ed-4e72-b75f-e449111e7751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:43.928277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618c8e3-20ed-4e72-b75f-e449111e7751 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:43.928567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618c8e3-20ed-4e72-b75f-e449111e7751 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:43.928853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618c8e3-20ed-4e72-b75f-e449111e7751 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:43.930619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2618c8e3-20ed-4e72-b75f-e449111e7751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:43.932114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2618c8e3-20ed-4e72-b75f-e449111e7751 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:43.933715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 30/55] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:45:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:43.939359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618c8e3-20ed-4e72-b75f-e449111e7751 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:43.942940 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:43.959659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50515ccf-cc2c-4375-aa44-cb7afe637f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:43.960858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50515ccf-cc2c-4375-aa44-cb7afe637f14 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:43.961149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50515ccf-cc2c-4375-aa44-cb7afe637f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:43.961433 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50515ccf-cc2c-4375-aa44-cb7afe637f14 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:43.962121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50515ccf-cc2c-4375-aa44-cb7afe637f14 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:43.963614 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50515ccf-cc2c-4375-aa44-cb7afe637f14 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:43.989090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 26/56] 10.222.0.46 () {60 vars in 1140 bytes} [Thu Jul 28 03:45:43 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:45:44.045325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618c8e3-20ed-4e72-b75f-e449111e7751 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:44.080374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c91ec-0157-4ddd-af7f-a9a07f438233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:44.081000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c91ec-0157-4ddd-af7f-a9a07f438233 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:44.081284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c91ec-0157-4ddd-af7f-a9a07f438233 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:44.081558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c91ec-0157-4ddd-af7f-a9a07f438233 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:44.082359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c91ec-0157-4ddd-af7f-a9a07f438233 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.domain.id=default, user.id=fe2947ccbc264fdba860153b59471d41) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:44.082667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c91ec-0157-4ddd-af7f-a9a07f438233 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:44.084163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c91ec-0157-4ddd-af7f-a9a07f438233 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:44.091162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 31/57] 10.222.0.46 () {56 vars in 1322 bytes} [Thu Jul 28 03:45:43 2022] GET /identity/v3/role_assignments?user.id=fe2947ccbc264fdba860153b59471d41&scope.domain.id=default&role.id=a37ae24c516b48c69e13721c013134a2 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:45.982234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38c744-0d71-40d5-a01d-5147636113f8 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:45.982745 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38c744-0d71-40d5-a01d-5147636113f8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:45.982875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38c744-0d71-40d5-a01d-5147636113f8 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:45.984181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 27/58] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:45.989139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cf821-0ff6-40e0-9d78-b7c53f9e4510 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:45.989546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cf821-0ff6-40e0-9d78-b7c53f9e4510 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:45.989711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cf821-0ff6-40e0-9d78-b7c53f9e4510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:46.008350 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-983cf821-0ff6-40e0-9d78-b7c53f9e4510 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:46.009531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-983cf821-0ff6-40e0-9d78-b7c53f9e4510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:46.069643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:46.075324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 32/59] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:46.099286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38c744-0d71-40d5-a01d-5147636113f8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:46.157041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:46.202844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38c744-0d71-40d5-a01d-5147636113f8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:46.236670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e4fc4-78a9-4384-9d1b-f6d7e8af3654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:46.237247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4fc4-78a9-4384-9d1b-f6d7e8af3654 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:46.237534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4fc4-78a9-4384-9d1b-f6d7e8af3654 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:46.237811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4fc4-78a9-4384-9d1b-f6d7e8af3654 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:46.238973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e4fc4-78a9-4384-9d1b-f6d7e8af3654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:46.240414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e4fc4-78a9-4384-9d1b-f6d7e8af3654 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:46.242600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 28/60] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:45:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:46.248072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cf821-0ff6-40e0-9d78-b7c53f9e4510 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:46.343372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cf821-0ff6-40e0-9d78-b7c53f9e4510 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:46.375992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b5568-c04a-4979-845a-bd08cc8a3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:46.376594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b5568-c04a-4979-845a-bd08cc8a3c4a None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:46.376875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b5568-c04a-4979-845a-bd08cc8a3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:46.377148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b5568-c04a-4979-845a-bd08cc8a3c4a None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:46.377828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b5568-c04a-4979-845a-bd08cc8a3c4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:46.379288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b5568-c04a-4979-845a-bd08cc8a3c4a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:46.404530 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 33/61] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:45:46 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:45:49.432139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa47a-5c9e-430f-807a-b374254edc4f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:49.432649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa47a-5c9e-430f-807a-b374254edc4f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:49.432757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa47a-5c9e-430f-807a-b374254edc4f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:49.434292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 29/62] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:49.452370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510e893-5f70-4be2-b45b-8406a6e2a382 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:49.452834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510e893-5f70-4be2-b45b-8406a6e2a382 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:49.453024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510e893-5f70-4be2-b45b-8406a6e2a382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:49.479851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3510e893-5f70-4be2-b45b-8406a6e2a382 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:49.481068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3510e893-5f70-4be2-b45b-8406a6e2a382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:49.548670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:49.578590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 34/63] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:49.614107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdaa47a-5c9e-430f-807a-b374254edc4f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:49.721237 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e458f-1936-4add-be89-b5b22bc4dcdb None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:49.721735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e458f-1936-4add-be89-b5b22bc4dcdb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:49.722157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e458f-1936-4add-be89-b5b22bc4dcdb None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:49.723668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 35/64] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:45:49.732893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d21c3-b2fd-442e-a29e-c878e62e118c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:49.733183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d21c3-b2fd-442e-a29e-c878e62e118c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:49.733448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d21c3-b2fd-442e-a29e-c878e62e118c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:49.737218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdaa47a-5c9e-430f-807a-b374254edc4f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:49.767486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:49.779653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ef2d21c3-b2fd-442e-a29e-c878e62e118c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:49.780860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ef2d21c3-b2fd-442e-a29e-c878e62e118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:49.782168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98804b34-5247-4fe7-8732-a83e17e73d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:49.782909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98804b34-5247-4fe7-8732-a83e17e73d4e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:49.783202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98804b34-5247-4fe7-8732-a83e17e73d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:49.783487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98804b34-5247-4fe7-8732-a83e17e73d4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:49.801932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98804b34-5247-4fe7-8732-a83e17e73d4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-98804b34-5247-4fe7-8732-a83e17e73d4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:49.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:49.809351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:45:49.810530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 30/65] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:45:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:49.828801 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98804b34-5247-4fe7-8732-a83e17e73d4e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:49.877336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 36/66] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:45:49.878686 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5ecb9-71f4-4749-a830-010072e54e1d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:49.879029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5ecb9-71f4-4749-a830-010072e54e1d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:49.879309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5ecb9-71f4-4749-a830-010072e54e1d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:49.880734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 37/67] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:49.887089 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7eb21-a099-4048-9001-bf611d53a77f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:49.887381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7eb21-a099-4048-9001-bf611d53a77f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:49.887662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7eb21-a099-4048-9001-bf611d53a77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:49.907033 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-dde7eb21-a099-4048-9001-bf611d53a77f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:49.908219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dde7eb21-a099-4048-9001-bf611d53a77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:49.951327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98804b34-5247-4fe7-8732-a83e17e73d4e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:49.988535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 38/68] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:45:49.989129 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb2cfd-7dbb-4e63-8e28-14eb4311c85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:49.989303 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7eb21-a099-4048-9001-bf611d53a77f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:49.989956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb2cfd-7dbb-4e63-8e28-14eb4311c85c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:49.990344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb2cfd-7dbb-4e63-8e28-14eb4311c85c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:49.990723 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb2cfd-7dbb-4e63-8e28-14eb4311c85c None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:49.991649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb2cfd-7dbb-4e63-8e28-14eb4311c85c None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:49.992243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb2cfd-7dbb-4e63-8e28-14eb4311c85c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:49.993747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb2cfd-7dbb-4e63-8e28-14eb4311c85c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:50.003030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 31/69] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:45:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:50.015973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb2cfd-7dbb-4e63-8e28-14eb4311c85c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:50.047628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:50.072757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:50.108061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7eb21-a099-4048-9001-bf611d53a77f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:50.132748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb2cfd-7dbb-4e63-8e28-14eb4311c85c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:50.144216 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67b931-d0e2-4f33-9141-daf295783e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:50.144869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67b931-d0e2-4f33-9141-daf295783e75 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.145157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67b931-d0e2-4f33-9141-daf295783e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.145435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67b931-d0e2-4f33-9141-daf295783e75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.157759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67b931-d0e2-4f33-9141-daf295783e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-8b67b931-d0e2-4f33-9141-daf295783e75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:50.163081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:50.163927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:45:50.164673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:45:50.170080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 39/70] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:45:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:50.170080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c703d-fb59-4b9a-ad32-315be05bdc04 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.170080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c703d-fb59-4b9a-ad32-315be05bdc04 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.170080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c703d-fb59-4b9a-ad32-315be05bdc04 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.170080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 40/71] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:50.170080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c703d-fb59-4b9a-ad32-315be05bdc04 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:50.177089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ff17-43f9-4d1b-b061-53aec655c81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:50.177751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ff17-43f9-4d1b-b061-53aec655c81d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.178040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ff17-43f9-4d1b-b061-53aec655c81d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.178320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ff17-43f9-4d1b-b061-53aec655c81d None admin] PATH_INFO: `/v3/users/5993366070364e2ea23f57f39e958895` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.195135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598ff17-43f9-4d1b-b061-53aec655c81d None admin] RBAC: Authorizing `identity:get_user(user_id=5993366070364e2ea23f57f39e958895)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:50.196668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598ff17-43f9-4d1b-b061-53aec655c81d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:50.198492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 32/72] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:50 2022] GET /identity/v3/users/5993366070364e2ea23f57f39e958895 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:50.199110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ff17-43f9-4d1b-b061-53aec655c81d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:50.274822 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c703d-fb59-4b9a-ad32-315be05bdc04 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:50.277010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:50.299688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ff17-43f9-4d1b-b061-53aec655c81d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:50.302248 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:50.310192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a05f8-6a72-45f6-a7ec-333417e1bf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:50.310890 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a05f8-6a72-45f6-a7ec-333417e1bf12 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.311178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a05f8-6a72-45f6-a7ec-333417e1bf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.311454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a05f8-6a72-45f6-a7ec-333417e1bf12 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.322654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a05f8-6a72-45f6-a7ec-333417e1bf12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-585a05f8-6a72-45f6-a7ec-333417e1bf12 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:50.327598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:50.328417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:45:50.329172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 41/73] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:45:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:50.330497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19528c6a-de8e-4145-84de-6284e4dabdf5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.330497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19528c6a-de8e-4145-84de-6284e4dabdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.330497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19528c6a-de8e-4145-84de-6284e4dabdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.334410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb908b7f-dc28-46b6-adf9-d2d2e35e56ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:50.335326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb908b7f-dc28-46b6-adf9-d2d2e35e56ab None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.335592 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb908b7f-dc28-46b6-adf9-d2d2e35e56ab None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.335878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb908b7f-dc28-46b6-adf9-d2d2e35e56ab None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.336637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb908b7f-dc28-46b6-adf9-d2d2e35e56ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:50.336950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb908b7f-dc28-46b6-adf9-d2d2e35e56ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:50.338344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb908b7f-dc28-46b6-adf9-d2d2e35e56ab None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:50.351779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 33/74] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:45:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:50.351779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d5309-1437-431d-8294-2ffcef90373e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.351779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d5309-1437-431d-8294-2ffcef90373e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.351779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d5309-1437-431d-8294-2ffcef90373e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.351779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 34/75] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:50.351779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d5309-1437-431d-8294-2ffcef90373e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:50.352810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-19528c6a-de8e-4145-84de-6284e4dabdf5 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:50.354116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-19528c6a-de8e-4145-84de-6284e4dabdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:50.420628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 42/76] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:50.421548 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19528c6a-de8e-4145-84de-6284e4dabdf5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:50.460649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d5309-1437-431d-8294-2ffcef90373e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:50.494801 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe733b-8e8b-4eea-91f2-9a16fa03c7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:50.495425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe733b-8e8b-4eea-91f2-9a16fa03c7cc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.495713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe733b-8e8b-4eea-91f2-9a16fa03c7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.495995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe733b-8e8b-4eea-91f2-9a16fa03c7cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.521158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:50.528689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19528c6a-de8e-4145-84de-6284e4dabdf5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:50.540046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:50.563914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e944bf-f68c-4dcb-bdf8-5142dfeef693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:50.564568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e944bf-f68c-4dcb-bdf8-5142dfeef693 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.564854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e944bf-f68c-4dcb-bdf8-5142dfeef693 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.565652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e944bf-f68c-4dcb-bdf8-5142dfeef693 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.566396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e944bf-f68c-4dcb-bdf8-5142dfeef693 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:50.566711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e944bf-f68c-4dcb-bdf8-5142dfeef693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:50.568134 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e944bf-f68c-4dcb-bdf8-5142dfeef693 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:50.577327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 43/77] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:45:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:50.578127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e944bf-f68c-4dcb-bdf8-5142dfeef693 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:50.580769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe733b-8e8b-4eea-91f2-9a16fa03c7cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:50.582309 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe733b-8e8b-4eea-91f2-9a16fa03c7cc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:50.676358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e944bf-f68c-4dcb-bdf8-5142dfeef693 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:50.681360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 35/78] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:45:50.682595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f47fd-c4f2-401d-9c3e-af5f8f39a283 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.682917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f47fd-c4f2-401d-9c3e-af5f8f39a283 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.683195 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f47fd-c4f2-401d-9c3e-af5f8f39a283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.700921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8b8f47fd-c4f2-401d-9c3e-af5f8f39a283 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:50.702112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8b8f47fd-c4f2-401d-9c3e-af5f8f39a283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:50.710970 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b1e46-d1b4-4e3d-b7de-6f29dc4fff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:50.711615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b1e46-d1b4-4e3d-b7de-6f29dc4fff1b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.711907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b1e46-d1b4-4e3d-b7de-6f29dc4fff1b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.712195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b1e46-d1b4-4e3d-b7de-6f29dc4fff1b None admin] PATH_INFO: `/v3/users/fe2947ccbc264fdba860153b59471d41` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.713372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b1e46-d1b4-4e3d-b7de-6f29dc4fff1b None admin] RBAC: Authorizing `identity:get_user(user_id=fe2947ccbc264fdba860153b59471d41)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:50.714857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b1e46-d1b4-4e3d-b7de-6f29dc4fff1b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:50.716458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 44/79] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:50 2022] GET /identity/v3/users/fe2947ccbc264fdba860153b59471d41 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:50.719707 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b1e46-d1b4-4e3d-b7de-6f29dc4fff1b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:50.744581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:50.761225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:50.768984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 36/80] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:50.769808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f47fd-c4f2-401d-9c3e-af5f8f39a283 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:50.823573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b1e46-d1b4-4e3d-b7de-6f29dc4fff1b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:50.856514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a7296-349a-41e1-8842-a015fbd97067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:50.857157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a7296-349a-41e1-8842-a015fbd97067 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.857440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a7296-349a-41e1-8842-a015fbd97067 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.857716 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a7296-349a-41e1-8842-a015fbd97067 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.868328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a7296-349a-41e1-8842-a015fbd97067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-dd1a7296-349a-41e1-8842-a015fbd97067 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:50.873555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:50.874418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:45:50.875366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 45/81] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:45:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:50.875985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a7296-349a-41e1-8842-a015fbd97067 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:50.886759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f47fd-c4f2-401d-9c3e-af5f8f39a283 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:50.921050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40e4b2-a268-40b4-a20c-701fcdccc06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:50.921686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40e4b2-a268-40b4-a20c-701fcdccc06d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:50.921967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40e4b2-a268-40b4-a20c-701fcdccc06d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:50.922245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40e4b2-a268-40b4-a20c-701fcdccc06d None admin] PATH_INFO: `/v3/users/5993366070364e2ea23f57f39e958895` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:50.923429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40e4b2-a268-40b4-a20c-701fcdccc06d None admin] RBAC: Authorizing `identity:get_user(user_id=5993366070364e2ea23f57f39e958895)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:50.924883 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40e4b2-a268-40b4-a20c-701fcdccc06d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:50.926473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 37/82] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:50 2022] GET /identity/v3/users/5993366070364e2ea23f57f39e958895 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:50.929996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40e4b2-a268-40b4-a20c-701fcdccc06d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:50.961063 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:50.978169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a7296-349a-41e1-8842-a015fbd97067 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:50.992179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:51.011724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe9953-d111-4a31-8cdd-def9bc31ff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:51.012362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9953-d111-4a31-8cdd-def9bc31ff8e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:51.012653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9953-d111-4a31-8cdd-def9bc31ff8e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:51.012940 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9953-d111-4a31-8cdd-def9bc31ff8e None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:51.014114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe9953-d111-4a31-8cdd-def9bc31ff8e None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:51.015591 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe9953-d111-4a31-8cdd-def9bc31ff8e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:51.017262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 46/83] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:45:50 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:51.018028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe9953-d111-4a31-8cdd-def9bc31ff8e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:51.030500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40e4b2-a268-40b4-a20c-701fcdccc06d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:51.064581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054db912-0203-4a63-848b-6f759bd4051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:51.065202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054db912-0203-4a63-848b-6f759bd4051d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:51.065489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054db912-0203-4a63-848b-6f759bd4051d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:51.065767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054db912-0203-4a63-848b-6f759bd4051d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:51.121782 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe9953-d111-4a31-8cdd-def9bc31ff8e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:51.131402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054db912-0203-4a63-848b-6f759bd4051d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:51.133155 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054db912-0203-4a63-848b-6f759bd4051d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:51.154737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaadd9-f3d6-4224-8d5d-3e747a4cb4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:51.155423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaadd9-f3d6-4224-8d5d-3e747a4cb4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:51.155714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaadd9-f3d6-4224-8d5d-3e747a4cb4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:51.155996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaadd9-f3d6-4224-8d5d-3e747a4cb4d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:51.157159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaadd9-f3d6-4224-8d5d-3e747a4cb4d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:51.158607 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaadd9-f3d6-4224-8d5d-3e747a4cb4d4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:51.160238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 47/84] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:45:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:51.161060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaadd9-f3d6-4224-8d5d-3e747a4cb4d4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:51.183606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:51.195982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:51.234756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 38/85] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:45:51.236711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054db912-0203-4a63-848b-6f759bd4051d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:51.262455 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaadd9-f3d6-4224-8d5d-3e747a4cb4d4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:51.296427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969602e-55df-4eb7-89c5-bad831fa9f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:51.297044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969602e-55df-4eb7-89c5-bad831fa9f7b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:51.297335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969602e-55df-4eb7-89c5-bad831fa9f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:51.297618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969602e-55df-4eb7-89c5-bad831fa9f7b None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:51.298337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969602e-55df-4eb7-89c5-bad831fa9f7b None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:51.298647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969602e-55df-4eb7-89c5-bad831fa9f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:51.300063 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969602e-55df-4eb7-89c5-bad831fa9f7b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:51.308639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 48/86] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:45:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:51.309428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969602e-55df-4eb7-89c5-bad831fa9f7b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:51.335497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054db912-0203-4a63-848b-6f759bd4051d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:51.369049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65fdbe-4e45-4632-b1db-d87b8fdda7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:51.369671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65fdbe-4e45-4632-b1db-d87b8fdda7cb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:51.369958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65fdbe-4e45-4632-b1db-d87b8fdda7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:51.370239 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65fdbe-4e45-4632-b1db-d87b8fdda7cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:51.392629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:51.400928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:51.411597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969602e-55df-4eb7-89c5-bad831fa9f7b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:51.438329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65fdbe-4e45-4632-b1db-d87b8fdda7cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:51.439831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65fdbe-4e45-4632-b1db-d87b8fdda7cb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:51.444876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa6b93-2d27-4317-86d4-0e308bce5e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:51.445558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa6b93-2d27-4317-86d4-0e308bce5e0e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:51.445852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa6b93-2d27-4317-86d4-0e308bce5e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:51.446138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa6b93-2d27-4317-86d4-0e308bce5e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:51.446971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa6b93-2d27-4317-86d4-0e308bce5e0e None admin] RBAC: Adding query filter params (role.id=f66cbbcc2491438184f04522c2a68578, scope.project.id=a75d68fb290c4a1494243d8817754029, user.id=5993366070364e2ea23f57f39e958895) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:51.447292 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa6b93-2d27-4317-86d4-0e308bce5e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:51.448787 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa6b93-2d27-4317-86d4-0e308bce5e0e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:51.454112 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 49/87] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:45:51 2022] GET /identity/v3/role_assignments?user.id=5993366070364e2ea23f57f39e958895&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=f66cbbcc2491438184f04522c2a68578 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:51.454966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa6b93-2d27-4317-86d4-0e308bce5e0e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:51.560673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 39/88] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:45:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:45:51.560673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65fdbe-4e45-4632-b1db-d87b8fdda7cb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:51.578757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa6b93-2d27-4317-86d4-0e308bce5e0e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:51.625778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520142e-bbb1-4de6-9ac8-6f38788d06f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:51.626415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520142e-bbb1-4de6-9ac8-6f38788d06f7 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:51.626701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520142e-bbb1-4de6-9ac8-6f38788d06f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:51.627022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520142e-bbb1-4de6-9ac8-6f38788d06f7 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:51.627722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520142e-bbb1-4de6-9ac8-6f38788d06f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:51.629188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520142e-bbb1-4de6-9ac8-6f38788d06f7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:51.634894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:51.656340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 50/89] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:45:51 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:45:51.656340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520142e-bbb1-4de6-9ac8-6f38788d06f7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:51.656340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:51.699135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65fdbe-4e45-4632-b1db-d87b8fdda7cb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:51.752353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87f233-e784-42fd-8841-b70467a81bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:51.753287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87f233-e784-42fd-8841-b70467a81bfc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:51.753725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87f233-e784-42fd-8841-b70467a81bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:51.754152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87f233-e784-42fd-8841-b70467a81bfc None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:51.756503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87f233-e784-42fd-8841-b70467a81bfc None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:51.758339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87f233-e784-42fd-8841-b70467a81bfc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:51.761144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 40/90] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:45:51 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:51.762139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87f233-e784-42fd-8841-b70467a81bfc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:51.794569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520142e-bbb1-4de6-9ac8-6f38788d06f7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:51.845504 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f292be-2eb1-42ee-b4a7-7b52cffed76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:51.846724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f292be-2eb1-42ee-b4a7-7b52cffed76c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:51.847158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f292be-2eb1-42ee-b4a7-7b52cffed76c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:51.847442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f292be-2eb1-42ee-b4a7-7b52cffed76c None admin] PATH_INFO: `/v3/users/5993366070364e2ea23f57f39e958895` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:51.849225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f292be-2eb1-42ee-b4a7-7b52cffed76c None admin] RBAC: Authorizing `identity:get_user(user_id=5993366070364e2ea23f57f39e958895)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:51.851420 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f292be-2eb1-42ee-b4a7-7b52cffed76c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:51.853120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 51/91] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:51 2022] GET /identity/v3/users/5993366070364e2ea23f57f39e958895 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:51.854975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f292be-2eb1-42ee-b4a7-7b52cffed76c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:51.906563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87f233-e784-42fd-8841-b70467a81bfc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:51.950723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:51.957366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:51.965885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8c734-1be1-41dc-8eb4-fa882df6e680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:51.966637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8c734-1be1-41dc-8eb4-fa882df6e680 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:51.967264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8c734-1be1-41dc-8eb4-fa882df6e680 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:51.967551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8c734-1be1-41dc-8eb4-fa882df6e680 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:51.969368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8c734-1be1-41dc-8eb4-fa882df6e680 None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:51.971224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8c734-1be1-41dc-8eb4-fa882df6e680 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:51.973626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 41/92] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:45:51 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:51.976130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8c734-1be1-41dc-8eb4-fa882df6e680 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:51.993972 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f292be-2eb1-42ee-b4a7-7b52cffed76c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:52.035507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcdc53-89e3-4cd8-a2c2-8f487d0e766e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:52.036545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcdc53-89e3-4cd8-a2c2-8f487d0e766e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:52.037097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcdc53-89e3-4cd8-a2c2-8f487d0e766e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:52.037370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcdc53-89e3-4cd8-a2c2-8f487d0e766e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:52.038893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbcdc53-89e3-4cd8-a2c2-8f487d0e766e None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=a75d68fb290c4a1494243d8817754029, user.id=fe2947ccbc264fdba860153b59471d41) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:52.039343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbcdc53-89e3-4cd8-a2c2-8f487d0e766e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:52.040832 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbcdc53-89e3-4cd8-a2c2-8f487d0e766e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:52.047302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 52/93] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:45:51 2022] GET /identity/v3/role_assignments?user.id=fe2947ccbc264fdba860153b59471d41&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=a37ae24c516b48c69e13721c013134a2 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:52.048079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcdc53-89e3-4cd8-a2c2-8f487d0e766e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:52.108767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8c734-1be1-41dc-8eb4-fa882df6e680 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:52.156320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88100a-84f9-487d-a8d1-d3b94aa2c51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:52.157535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88100a-84f9-487d-a8d1-d3b94aa2c51f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:52.157862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88100a-84f9-487d-a8d1-d3b94aa2c51f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:52.158240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88100a-84f9-487d-a8d1-d3b94aa2c51f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:52.193207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcdc53-89e3-4cd8-a2c2-8f487d0e766e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:52.242101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:52.245269 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88100a-84f9-487d-a8d1-d3b94aa2c51f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:52.247060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88100a-84f9-487d-a8d1-d3b94aa2c51f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:52.263043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c6f31-219a-4a62-9051-e08b3a1d4e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:52.264432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c6f31-219a-4a62-9051-e08b3a1d4e54 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:52.265092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c6f31-219a-4a62-9051-e08b3a1d4e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:52.265712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c6f31-219a-4a62-9051-e08b3a1d4e54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:52.267142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c6f31-219a-4a62-9051-e08b3a1d4e54 None admin] RBAC: Adding query filter params (role.id=854a7f7c663b4c66a08f7e731c7bf63c, scope.project.id=a75d68fb290c4a1494243d8817754029, user.id=5993366070364e2ea23f57f39e958895) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:52.267599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c6f31-219a-4a62-9051-e08b3a1d4e54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:52.276267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c6f31-219a-4a62-9051-e08b3a1d4e54 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:52.278607 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:52.291567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 53/94] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:45:52 2022] GET /identity/v3/role_assignments?user.id=5993366070364e2ea23f57f39e958895&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=854a7f7c663b4c66a08f7e731c7bf63c => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:52.363965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 42/95] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:45:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:45:52.370522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c6f31-219a-4a62-9051-e08b3a1d4e54 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:52.524179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c6f31-219a-4a62-9051-e08b3a1d4e54 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:52.571560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f980d-5a71-4e72-b624-799ee5256ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:52.572235 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f980d-5a71-4e72-b624-799ee5256ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:52.572520 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f980d-5a71-4e72-b624-799ee5256ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:52.572799 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f980d-5a71-4e72-b624-799ee5256ec6 None admin] PATH_INFO: `/v3/projects/d59be85cc0f8416f952e6e214a0aa96f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:52.574043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f980d-5a71-4e72-b624-799ee5256ec6 None admin] RBAC: Authorizing `identity:get_project(project_id=d59be85cc0f8416f952e6e214a0aa96f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:52.575552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f980d-5a71-4e72-b624-799ee5256ec6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:52.577237 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 54/96] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:45:52 2022] GET /identity/v3/projects/d59be85cc0f8416f952e6e214a0aa96f => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:52.583273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88100a-84f9-487d-a8d1-d3b94aa2c51f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:52.700232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:52.703794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88100a-84f9-487d-a8d1-d3b94aa2c51f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:52.746335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144eb63-fd42-4872-82bf-704ce0b484a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:52.746989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144eb63-fd42-4872-82bf-704ce0b484a1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:52.747274 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144eb63-fd42-4872-82bf-704ce0b484a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:52.747550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144eb63-fd42-4872-82bf-704ce0b484a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:52.748359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144eb63-fd42-4872-82bf-704ce0b484a1 None admin] RBAC: Adding query filter params (role.id=f66cbbcc2491438184f04522c2a68578, scope.project.id=d59be85cc0f8416f952e6e214a0aa96f, user.id=5993366070364e2ea23f57f39e958895) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:52.748670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144eb63-fd42-4872-82bf-704ce0b484a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:52.750122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144eb63-fd42-4872-82bf-704ce0b484a1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:52.759345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 43/97] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:45:52 2022] GET /identity/v3/role_assignments?user.id=5993366070364e2ea23f57f39e958895&scope.project.id=d59be85cc0f8416f952e6e214a0aa96f&role.id=f66cbbcc2491438184f04522c2a68578 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:54.062217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01e5ab-d1b8-4c74-b2fe-e34a7cda18c9 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:54.063269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01e5ab-d1b8-4c74-b2fe-e34a7cda18c9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:54.063763 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01e5ab-d1b8-4c74-b2fe-e34a7cda18c9 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:54.065414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 55/98] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:54.071790 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea3e9-2408-46c1-871f-5fd27566ec21 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:54.072220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea3e9-2408-46c1-871f-5fd27566ec21 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:54.072387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea3e9-2408-46c1-871f-5fd27566ec21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:54.099738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-70bea3e9-2408-46c1-871f-5fd27566ec21 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:54.101011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-70bea3e9-2408-46c1-871f-5fd27566ec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:54.185441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 44/99] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:45:54.228295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01e5ab-d1b8-4c74-b2fe-e34a7cda18c9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:54.315961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:54.354426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01e5ab-d1b8-4c74-b2fe-e34a7cda18c9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:54.396650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c30c41-e4fc-4dd0-8791-ff9dd8e785e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:54.398318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30c41-e4fc-4dd0-8791-ff9dd8e785e4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:54.398825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30c41-e4fc-4dd0-8791-ff9dd8e785e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:54.399298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30c41-e4fc-4dd0-8791-ff9dd8e785e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:54.410307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c30c41-e4fc-4dd0-8791-ff9dd8e785e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-b8c30c41-e4fc-4dd0-8791-ff9dd8e785e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:54.418276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:54.419645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:45:54.420812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:45:54.423537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 56/100] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:45:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:54.429524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bea3e9-2408-46c1-871f-5fd27566ec21 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:54.558736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:54.569557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bea3e9-2408-46c1-871f-5fd27566ec21 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:54.616521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19a44f-0b19-4a7c-8ff6-492f4c945192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:54.617163 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19a44f-0b19-4a7c-8ff6-492f4c945192 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:54.617451 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19a44f-0b19-4a7c-8ff6-492f4c945192 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:54.617733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19a44f-0b19-4a7c-8ff6-492f4c945192 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:54.618460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19a44f-0b19-4a7c-8ff6-492f4c945192 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:54.618772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19a44f-0b19-4a7c-8ff6-492f4c945192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:54.620185 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19a44f-0b19-4a7c-8ff6-492f4c945192 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:54.629275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 45/101] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:45:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:54.634637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c30c41-e4fc-4dd0-8791-ff9dd8e785e4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:54.779244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c30c41-e4fc-4dd0-8791-ff9dd8e785e4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:54.819070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:54.829302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afce64b-da8f-4cd6-b380-434b47102754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:54.830279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce64b-da8f-4cd6-b380-434b47102754 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:54.830829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce64b-da8f-4cd6-b380-434b47102754 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:54.831307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce64b-da8f-4cd6-b380-434b47102754 None admin] PATH_INFO: `/v3/users/5993366070364e2ea23f57f39e958895` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:54.832726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce64b-da8f-4cd6-b380-434b47102754 None admin] RBAC: Authorizing `identity:get_user(user_id=5993366070364e2ea23f57f39e958895)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:54.834566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce64b-da8f-4cd6-b380-434b47102754 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:54.836977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 57/102] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:54 2022] GET /identity/v3/users/5993366070364e2ea23f57f39e958895 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:54.842705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19a44f-0b19-4a7c-8ff6-492f4c945192 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:54.890669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcedb5-67df-43f5-b13a-40cc8d6f164a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:54.891309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcedb5-67df-43f5-b13a-40cc8d6f164a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:54.891781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcedb5-67df-43f5-b13a-40cc8d6f164a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:54.893329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 58/103] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:54.898988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9a7de-4476-4e14-92be-3bc77232c2a4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:54.899523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9a7de-4476-4e14-92be-3bc77232c2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:54.899998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9a7de-4476-4e14-92be-3bc77232c2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:54.929398 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e5f9a7de-4476-4e14-92be-3bc77232c2a4 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:54.930620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e5f9a7de-4476-4e14-92be-3bc77232c2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:54.967307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19a44f-0b19-4a7c-8ff6-492f4c945192 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:55.012280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a6520-d3f6-4fc7-86eb-8c319f94c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:55.012945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a6520-d3f6-4fc7-86eb-8c319f94c515 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.013246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a6520-d3f6-4fc7-86eb-8c319f94c515 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.013508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a6520-d3f6-4fc7-86eb-8c319f94c515 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.015508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a6520-d3f6-4fc7-86eb-8c319f94c515 None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:55.016972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a6520-d3f6-4fc7-86eb-8c319f94c515 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:55.025190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 46/104] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:45:54 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:55.026538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d98cc-60e9-4a70-a074-530838576ec2 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.026845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d98cc-60e9-4a70-a074-530838576ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.027125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d98cc-60e9-4a70-a074-530838576ec2 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.028430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 47/105] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:55.029109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d98cc-60e9-4a70-a074-530838576ec2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:55.055484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 59/106] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:45:55.056794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965b09-c086-4d9a-9251-8e8aa6035c9a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.057083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965b09-c086-4d9a-9251-8e8aa6035c9a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.057359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965b09-c086-4d9a-9251-8e8aa6035c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.088246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:55.100102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2d965b09-c086-4d9a-9251-8e8aa6035c9a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:55.102483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2d965b09-c086-4d9a-9251-8e8aa6035c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:55.181093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d98cc-60e9-4a70-a074-530838576ec2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:55.220706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf80b9f-7228-4f25-b7c8-1b89fa2230c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:55.221499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf80b9f-7228-4f25-b7c8-1b89fa2230c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.221788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf80b9f-7228-4f25-b7c8-1b89fa2230c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.222064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf80b9f-7228-4f25-b7c8-1b89fa2230c4 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029/users/5993366070364e2ea23f57f39e958895/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.227384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf80b9f-7228-4f25-b7c8-1b89fa2230c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a75d68fb290c4a1494243d8817754029, user_id=5993366070364e2ea23f57f39e958895, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:55.228828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf80b9f-7228-4f25-b7c8-1b89fa2230c4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:55.230983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 60/107] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:55.232906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d965b09-c086-4d9a-9251-8e8aa6035c9a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:55.258615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 48/108] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:45:55 2022] PUT /identity/v3/projects/a75d68fb290c4a1494243d8817754029/users/5993366070364e2ea23f57f39e958895/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:45:55.259307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf80b9f-7228-4f25-b7c8-1b89fa2230c4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:55.369059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:55.422217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:55.453934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf80b9f-7228-4f25-b7c8-1b89fa2230c4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:55.458503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d965b09-c086-4d9a-9251-8e8aa6035c9a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:55.514776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563abae-bae5-48bf-814e-e396846681e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:55.515036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda3f3c-9d15-433a-9c22-a934bce39fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:55.515532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563abae-bae5-48bf-814e-e396846681e8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.515706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda3f3c-9d15-433a-9c22-a934bce39fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.515857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563abae-bae5-48bf-814e-e396846681e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.515945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda3f3c-9d15-433a-9c22-a934bce39fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.516573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563abae-bae5-48bf-814e-e396846681e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.516680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda3f3c-9d15-433a-9c22-a934bce39fc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.541139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda3f3c-9d15-433a-9c22-a934bce39fc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1dda3f3c-9d15-433a-9c22-a934bce39fc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:55.546399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:55.547298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:45:55.548474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 61/109] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:45:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:55.549331 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563abae-bae5-48bf-814e-e396846681e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:55.549401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936c5b-f57c-4596-abff-6ee97438ab83 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.555437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936c5b-f57c-4596-abff-6ee97438ab83 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.555767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936c5b-f57c-4596-abff-6ee97438ab83 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2563abae-bae5-48bf-814e-e396846681e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:55.556900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:55.559013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:45:55.561068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:45:55.562468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 62/110] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:55.562468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0936c5b-f57c-4596-abff-6ee97438ab83 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:55.562734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 49/111] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:45:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:55.568369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d1859-33f8-43ad-b77c-e7dc8e9db9e8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.568720 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d1859-33f8-43ad-b77c-e7dc8e9db9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.568991 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d1859-33f8-43ad-b77c-e7dc8e9db9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.588682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c01d1859-33f8-43ad-b77c-e7dc8e9db9e8 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:55.589996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c01d1859-33f8-43ad-b77c-e7dc8e9db9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:55.665181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 50/112] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:55.666022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d1859-33f8-43ad-b77c-e7dc8e9db9e8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:55.697665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0936c5b-f57c-4596-abff-6ee97438ab83 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:55.716951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:55.750338 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220bc60-ad11-4d1d-8264-dd3098a4712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:55.750991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220bc60-ad11-4d1d-8264-dd3098a4712e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.751280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220bc60-ad11-4d1d-8264-dd3098a4712e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.751558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220bc60-ad11-4d1d-8264-dd3098a4712e None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.752288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220bc60-ad11-4d1d-8264-dd3098a4712e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:55.752594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220bc60-ad11-4d1d-8264-dd3098a4712e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:55.753554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:55.753982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220bc60-ad11-4d1d-8264-dd3098a4712e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:55.762568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 63/113] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:45:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:55.763826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4e0f-472a-4736-80da-738b29594434 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.764115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4e0f-472a-4736-80da-738b29594434 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.764394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4e0f-472a-4736-80da-738b29594434 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.765739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 64/114] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:55.766295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f4e0f-472a-4736-80da-738b29594434 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:55.788841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d1859-33f8-43ad-b77c-e7dc8e9db9e8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:55.825540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae455f30-6b47-4a89-8bc6-70226f65b158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:55.826465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae455f30-6b47-4a89-8bc6-70226f65b158 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.827067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae455f30-6b47-4a89-8bc6-70226f65b158 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.827421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae455f30-6b47-4a89-8bc6-70226f65b158 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.828273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae455f30-6b47-4a89-8bc6-70226f65b158 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:55.829896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae455f30-6b47-4a89-8bc6-70226f65b158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:55.829990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae455f30-6b47-4a89-8bc6-70226f65b158 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:55.843411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 51/115] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:45:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:55.843411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26025514-799e-44ef-b6ff-8c970ddd3813 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.843411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26025514-799e-44ef-b6ff-8c970ddd3813 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.843411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26025514-799e-44ef-b6ff-8c970ddd3813 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.843411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 52/116] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:55.843411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26025514-799e-44ef-b6ff-8c970ddd3813 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:55.886566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f4e0f-472a-4736-80da-738b29594434 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:55.923201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3f748-f597-4956-8288-859498f20184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:55.923770 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3f748-f597-4956-8288-859498f20184 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.924055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3f748-f597-4956-8288-859498f20184 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.924332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3f748-f597-4956-8288-859498f20184 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.925904 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3f748-f597-4956-8288-859498f20184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:55.927753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3f748-f597-4956-8288-859498f20184 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:55.930383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 65/117] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:45:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:55.931360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d751c9-0ca7-4a29-9297-b8588a932b7e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.931646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d751c9-0ca7-4a29-9297-b8588a932b7e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.931925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d751c9-0ca7-4a29-9297-b8588a932b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.949799 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c1d751c9-0ca7-4a29-9297-b8588a932b7e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:55.951009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c1d751c9-0ca7-4a29-9297-b8588a932b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:55.957757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26025514-799e-44ef-b6ff-8c970ddd3813 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:55.991913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:55.993859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78834c4e-788b-46b9-95fc-01119b51bd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:55.994572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834c4e-788b-46b9-95fc-01119b51bd8a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:55.994893 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834c4e-788b-46b9-95fc-01119b51bd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:55.995286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834c4e-788b-46b9-95fc-01119b51bd8a None admin] PATH_INFO: `/v3/users/5993366070364e2ea23f57f39e958895` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:55.996492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78834c4e-788b-46b9-95fc-01119b51bd8a None admin] RBAC: Authorizing `identity:get_user(user_id=5993366070364e2ea23f57f39e958895)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:55.997960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78834c4e-788b-46b9-95fc-01119b51bd8a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:55.999614 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 53/118] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:55 2022] GET /identity/v3/users/5993366070364e2ea23f57f39e958895 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:56.004710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78834c4e-788b-46b9-95fc-01119b51bd8a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:56.010161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:56.022358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 66/119] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:56.026819 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee605f-5e6b-4dfe-8256-91f574f15919 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:56.026819 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee605f-5e6b-4dfe-8256-91f574f15919 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:56.026819 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee605f-5e6b-4dfe-8256-91f574f15919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:56.046272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5dee605f-5e6b-4dfe-8256-91f574f15919 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:56.047492 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5dee605f-5e6b-4dfe-8256-91f574f15919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:56.119706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78834c4e-788b-46b9-95fc-01119b51bd8a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:56.126865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 67/120] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:56.126865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee605f-5e6b-4dfe-8256-91f574f15919 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:56.162641 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e90ef-22d7-4456-bdab-b3945b122edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:56.163866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e90ef-22d7-4456-bdab-b3945b122edb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:56.164152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e90ef-22d7-4456-bdab-b3945b122edb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:56.164431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e90ef-22d7-4456-bdab-b3945b122edb None admin] PATH_INFO: `/v3/users/fe2947ccbc264fdba860153b59471d41` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:56.165617 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e90ef-22d7-4456-bdab-b3945b122edb None admin] RBAC: Authorizing `identity:get_user(user_id=fe2947ccbc264fdba860153b59471d41)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:56.167175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e90ef-22d7-4456-bdab-b3945b122edb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:56.168880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 54/121] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:55 2022] GET /identity/v3/users/fe2947ccbc264fdba860153b59471d41 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:56.173904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e90ef-22d7-4456-bdab-b3945b122edb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:56.243549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee605f-5e6b-4dfe-8256-91f574f15919 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:56.246153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:56.249746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:56.281550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e2731-39eb-4e91-ba67-9145501fc6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:56.282177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e90ef-22d7-4456-bdab-b3945b122edb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:56.282289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e2731-39eb-4e91-ba67-9145501fc6a2 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:56.282469 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e2731-39eb-4e91-ba67-9145501fc6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:56.282736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e2731-39eb-4e91-ba67-9145501fc6a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:56.283447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e2731-39eb-4e91-ba67-9145501fc6a2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:56.284909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e2731-39eb-4e91-ba67-9145501fc6a2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:56.313236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 68/122] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:45:56 2022] POST /identity/v3/projects => generated 302 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:45:56.318118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e2731-39eb-4e91-ba67-9145501fc6a2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:56.329235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77337e86-d3b8-49df-8ccf-d02aeb10a2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:56.330056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77337e86-d3b8-49df-8ccf-d02aeb10a2fb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:56.330464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77337e86-d3b8-49df-8ccf-d02aeb10a2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:56.331201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77337e86-d3b8-49df-8ccf-d02aeb10a2fb None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:56.332319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77337e86-d3b8-49df-8ccf-d02aeb10a2fb None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:56.334062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77337e86-d3b8-49df-8ccf-d02aeb10a2fb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:56.337318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 55/123] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:45:56 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:56.338287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77337e86-d3b8-49df-8ccf-d02aeb10a2fb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:56.448661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e2731-39eb-4e91-ba67-9145501fc6a2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:56.480236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77337e86-d3b8-49df-8ccf-d02aeb10a2fb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:56.493783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dad93a-9787-45c7-826c-1fd9eeaff10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:56.494498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dad93a-9787-45c7-826c-1fd9eeaff10f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:56.494808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dad93a-9787-45c7-826c-1fd9eeaff10f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:56.495167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dad93a-9787-45c7-826c-1fd9eeaff10f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:56.508810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dad93a-9787-45c7-826c-1fd9eeaff10f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-f1dad93a-9787-45c7-826c-1fd9eeaff10f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:56.514266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:56.515146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:45:56.515947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 69/124] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:45:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:56.516572 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dad93a-9787-45c7-826c-1fd9eeaff10f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:56.529694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a901d47-76c4-45b0-bddf-9a318b2d1a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:56.530349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a901d47-76c4-45b0-bddf-9a318b2d1a60 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:56.530637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a901d47-76c4-45b0-bddf-9a318b2d1a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:56.533936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a901d47-76c4-45b0-bddf-9a318b2d1a60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:56.545024 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:56.545473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:56.549371 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a901d47-76c4-45b0-bddf-9a318b2d1a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5a901d47-76c4-45b0-bddf-9a318b2d1a60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:56.559933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:56.560836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 56/125] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:45:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:56.561994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a901d47-76c4-45b0-bddf-9a318b2d1a60 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:56.665124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dad93a-9787-45c7-826c-1fd9eeaff10f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:56.678019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a901d47-76c4-45b0-bddf-9a318b2d1a60 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:56.707760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369957c-5e07-4efb-9b13-7d903113c4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:56.708502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369957c-5e07-4efb-9b13-7d903113c4cb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:56.708836 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369957c-5e07-4efb-9b13-7d903113c4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:56.709438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369957c-5e07-4efb-9b13-7d903113c4cb None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:56.711134 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369957c-5e07-4efb-9b13-7d903113c4cb None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:56.713361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369957c-5e07-4efb-9b13-7d903113c4cb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:56.715348 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 70/126] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:45:56 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:56.716199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369957c-5e07-4efb-9b13-7d903113c4cb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:56.721950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae7214-144f-43c6-a8f1-d466b35eb4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:56.722630 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae7214-144f-43c6-a8f1-d466b35eb4c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:56.723135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae7214-144f-43c6-a8f1-d466b35eb4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:56.723425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae7214-144f-43c6-a8f1-d466b35eb4c2 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029/users/5993366070364e2ea23f57f39e958895/roles/854a7f7c663b4c66a08f7e731c7bf63c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:56.726803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae7214-144f-43c6-a8f1-d466b35eb4c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a75d68fb290c4a1494243d8817754029, user_id=5993366070364e2ea23f57f39e958895, role_id=854a7f7c663b4c66a08f7e731c7bf63c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:56.728273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae7214-144f-43c6-a8f1-d466b35eb4c2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:56.748102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 57/127] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:45:56 2022] PUT /identity/v3/projects/a75d68fb290c4a1494243d8817754029/users/5993366070364e2ea23f57f39e958895/roles/854a7f7c663b4c66a08f7e731c7bf63c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:45:56.748689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae7214-144f-43c6-a8f1-d466b35eb4c2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:56.888619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:56.897939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:56.921306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369957c-5e07-4efb-9b13-7d903113c4cb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:56.962280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae7214-144f-43c6-a8f1-d466b35eb4c2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:56.973142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f325d6b-7fe2-45ac-b826-8ea4d6e1424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:56.974235 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f325d6b-7fe2-45ac-b826-8ea4d6e1424d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:56.974542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f325d6b-7fe2-45ac-b826-8ea4d6e1424d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:56.974872 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f325d6b-7fe2-45ac-b826-8ea4d6e1424d None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:56.976347 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f325d6b-7fe2-45ac-b826-8ea4d6e1424d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:56.976730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f325d6b-7fe2-45ac-b826-8ea4d6e1424d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:56.978814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f325d6b-7fe2-45ac-b826-8ea4d6e1424d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:56.989731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 71/128] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:45:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:56.990726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f325d6b-7fe2-45ac-b826-8ea4d6e1424d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:57.000455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e6cb5-5058-45bc-8a6c-df381b8c8237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:57.001081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e6cb5-5058-45bc-8a6c-df381b8c8237 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:57.001362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e6cb5-5058-45bc-8a6c-df381b8c8237 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:57.001642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e6cb5-5058-45bc-8a6c-df381b8c8237 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:57.002425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e6cb5-5058-45bc-8a6c-df381b8c8237 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:57.002736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e6cb5-5058-45bc-8a6c-df381b8c8237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:57.004275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e6cb5-5058-45bc-8a6c-df381b8c8237 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:57.014993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 58/129] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:45:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:57.015623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e6cb5-5058-45bc-8a6c-df381b8c8237 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:57.127161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f325d6b-7fe2-45ac-b826-8ea4d6e1424d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:57.142931 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e6cb5-5058-45bc-8a6c-df381b8c8237 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:57.191158 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:57.192600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703ea51-4948-4468-b59c-a4b6a8bc7a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:57.194772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703ea51-4948-4468-b59c-a4b6a8bc7a59 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:57.195174 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703ea51-4948-4468-b59c-a4b6a8bc7a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:57.195509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703ea51-4948-4468-b59c-a4b6a8bc7a59 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029/users/fe2947ccbc264fdba860153b59471d41/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:57.198307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703ea51-4948-4468-b59c-a4b6a8bc7a59 None admin] RBAC: Authorizing `identity:create_grant(project_id=a75d68fb290c4a1494243d8817754029, user_id=fe2947ccbc264fdba860153b59471d41, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:57.202253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703ea51-4948-4468-b59c-a4b6a8bc7a59 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:57.215988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5891b1-c3c2-4fd2-8d2e-d1274c5e6309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:57.216658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5891b1-c3c2-4fd2-8d2e-d1274c5e6309 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:57.216946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5891b1-c3c2-4fd2-8d2e-d1274c5e6309 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:57.217221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5891b1-c3c2-4fd2-8d2e-d1274c5e6309 None admin] PATH_INFO: `/v3/users/76cc90df40644f83ba6c7346544ddfec` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:57.230796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 72/130] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:45:56 2022] PUT /identity/v3/projects/a75d68fb290c4a1494243d8817754029/users/fe2947ccbc264fdba860153b59471d41/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:45:57.233053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703ea51-4948-4468-b59c-a4b6a8bc7a59 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:57.251464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5891b1-c3c2-4fd2-8d2e-d1274c5e6309 None admin] RBAC: Authorizing `identity:get_user(user_id=76cc90df40644f83ba6c7346544ddfec)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:57.253252 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:57.253430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5891b1-c3c2-4fd2-8d2e-d1274c5e6309 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:57.258128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 59/131] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:57 2022] GET /identity/v3/users/76cc90df40644f83ba6c7346544ddfec => generated 289 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:57.264421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5891b1-c3c2-4fd2-8d2e-d1274c5e6309 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:57.407926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5891b1-c3c2-4fd2-8d2e-d1274c5e6309 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:57.460513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703ea51-4948-4468-b59c-a4b6a8bc7a59 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:57.474020 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225c0be-ff9a-42dd-a0e0-148264bfe8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:57.474575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225c0be-ff9a-42dd-a0e0-148264bfe8ac None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:57.474886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225c0be-ff9a-42dd-a0e0-148264bfe8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:57.475165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225c0be-ff9a-42dd-a0e0-148264bfe8ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:57.506430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fdad2-4a01-4160-9e6b-1e06fc0d2de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:57.507695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fdad2-4a01-4160-9e6b-1e06fc0d2de4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:57.507983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fdad2-4a01-4160-9e6b-1e06fc0d2de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:57.508526 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fdad2-4a01-4160-9e6b-1e06fc0d2de4 None admin] PATH_INFO: `/v3/users/5993366070364e2ea23f57f39e958895` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:57.510589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fdad2-4a01-4160-9e6b-1e06fc0d2de4 None admin] RBAC: Authorizing `identity:get_user(user_id=5993366070364e2ea23f57f39e958895)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:57.512121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fdad2-4a01-4160-9e6b-1e06fc0d2de4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:57.515487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 73/132] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:57 2022] GET /identity/v3/users/5993366070364e2ea23f57f39e958895 => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:57.516760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68deb6c-a2dd-406f-8c20-1cc1d7af3464 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:57.517038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68deb6c-a2dd-406f-8c20-1cc1d7af3464 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:57.517312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68deb6c-a2dd-406f-8c20-1cc1d7af3464 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:57.519381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 74/133] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:57.523790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68deb6c-a2dd-406f-8c20-1cc1d7af3464 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:57.533086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:57.578764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225c0be-ff9a-42dd-a0e0-148264bfe8ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:57.580284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225c0be-ff9a-42dd-a0e0-148264bfe8ac None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:57.632452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:57.675071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68deb6c-a2dd-406f-8c20-1cc1d7af3464 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:57.744774 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e496c7d-c094-48fe-ac42-7c0c2c0a2eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:57.747187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e496c7d-c094-48fe-ac42-7c0c2c0a2eca None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:57.747472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e496c7d-c094-48fe-ac42-7c0c2c0a2eca None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:57.747746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e496c7d-c094-48fe-ac42-7c0c2c0a2eca None admin] PATH_INFO: `/v3/projects/d59be85cc0f8416f952e6e214a0aa96f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:57.749033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 60/134] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:45:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:45:57.750301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4560d4-a140-40ee-9f2b-f930dc115b8b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:57.750589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4560d4-a140-40ee-9f2b-f930dc115b8b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:57.750900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4560d4-a140-40ee-9f2b-f930dc115b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:57.752898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e496c7d-c094-48fe-ac42-7c0c2c0a2eca None admin] RBAC: Authorizing `identity:get_project(project_id=d59be85cc0f8416f952e6e214a0aa96f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:57.754408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e496c7d-c094-48fe-ac42-7c0c2c0a2eca None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:57.757440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 75/135] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:45:57 2022] GET /identity/v3/projects/d59be85cc0f8416f952e6e214a0aa96f => generated 312 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:57.758143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e496c7d-c094-48fe-ac42-7c0c2c0a2eca None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:57.793996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ab4560d4-a140-40ee-9f2b-f930dc115b8b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:57.797064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ab4560d4-a140-40ee-9f2b-f930dc115b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:57.925495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 61/136] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:45:57.926364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4560d4-a140-40ee-9f2b-f930dc115b8b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:57.932447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e496c7d-c094-48fe-ac42-7c0c2c0a2eca None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:57.950482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:57.977468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ba88d-b00b-4ee7-8224-fdc608fbee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:57.978123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ba88d-b00b-4ee7-8224-fdc608fbee28 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:57.978405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ba88d-b00b-4ee7-8224-fdc608fbee28 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:57.978680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ba88d-b00b-4ee7-8224-fdc608fbee28 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:57.980567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ba88d-b00b-4ee7-8224-fdc608fbee28 None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:57.982030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ba88d-b00b-4ee7-8224-fdc608fbee28 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:57.983728 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 76/137] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:45:57 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:57.984499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ba88d-b00b-4ee7-8224-fdc608fbee28 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:58.016976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:58.128371 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4560d4-a140-40ee-9f2b-f930dc115b8b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:58.136564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ba88d-b00b-4ee7-8224-fdc608fbee28 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:58.172794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a47a0-42be-4a87-9794-c3cd70511101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:58.173520 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a47a0-42be-4a87-9794-c3cd70511101 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:58.173906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a47a0-42be-4a87-9794-c3cd70511101 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:58.174184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a47a0-42be-4a87-9794-c3cd70511101 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:58.174956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277f736-3ed1-4c00-9784-f5b86f91a67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:58.175685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277f736-3ed1-4c00-9784-f5b86f91a67a None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:58.176054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277f736-3ed1-4c00-9784-f5b86f91a67a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:58.176334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277f736-3ed1-4c00-9784-f5b86f91a67a None admin] PATH_INFO: `/v3/projects/d59be85cc0f8416f952e6e214a0aa96f/users/5993366070364e2ea23f57f39e958895/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:58.178993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277f736-3ed1-4c00-9784-f5b86f91a67a None admin] RBAC: Authorizing `identity:create_grant(project_id=d59be85cc0f8416f952e6e214a0aa96f, user_id=5993366070364e2ea23f57f39e958895, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:58.180468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277f736-3ed1-4c00-9784-f5b86f91a67a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:58.186176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a47a0-42be-4a87-9794-c3cd70511101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-c23a47a0-42be-4a87-9794-c3cd70511101 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:45:58.192058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:45:58.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 77/138] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:45:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:45:58.196634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a47a0-42be-4a87-9794-c3cd70511101 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:58.199525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 62/139] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:45:57 2022] PUT /identity/v3/projects/d59be85cc0f8416f952e6e214a0aa96f/users/5993366070364e2ea23f57f39e958895/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:45:58.206824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277f736-3ed1-4c00-9784-f5b86f91a67a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:58.373958 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:58.401911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:58.453073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a47a0-42be-4a87-9794-c3cd70511101 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:58.493545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277f736-3ed1-4c00-9784-f5b86f91a67a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:58.504808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166a75a-9e81-48dd-b65e-a98f14b69334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:58.510807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166a75a-9e81-48dd-b65e-a98f14b69334 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:58.510807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166a75a-9e81-48dd-b65e-a98f14b69334 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:58.510807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166a75a-9e81-48dd-b65e-a98f14b69334 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:58.510807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166a75a-9e81-48dd-b65e-a98f14b69334 None admin] RBAC: Adding query filter params (role.id=77b5c5dec7334e27a8126a96b7b17f32, scope.project.id=a75d68fb290c4a1494243d8817754029, user.id=76cc90df40644f83ba6c7346544ddfec) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:58.510807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166a75a-9e81-48dd-b65e-a98f14b69334 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:58.510807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166a75a-9e81-48dd-b65e-a98f14b69334 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:58.517238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 78/140] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:45:58 2022] GET /identity/v3/role_assignments?user.id=76cc90df40644f83ba6c7346544ddfec&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=77b5c5dec7334e27a8126a96b7b17f32 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:58.561476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7a6b6-2198-4794-9ac6-0790d7657509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:58.562185 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a6b6-2198-4794-9ac6-0790d7657509 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:58.562470 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a6b6-2198-4794-9ac6-0790d7657509 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:58.562744 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a6b6-2198-4794-9ac6-0790d7657509 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:58.564007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7a6b6-2198-4794-9ac6-0790d7657509 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:58.564561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7a6b6-2198-4794-9ac6-0790d7657509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:58.566991 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7a6b6-2198-4794-9ac6-0790d7657509 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:58.578832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 63/141] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:45:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:58.584206 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166a75a-9e81-48dd-b65e-a98f14b69334 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:58.680554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25614b23-bb61-456b-a842-4bde261ffc44 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:58.680847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25614b23-bb61-456b-a842-4bde261ffc44 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:58.681032 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25614b23-bb61-456b-a842-4bde261ffc44 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:58.682596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 64/142] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:45:58.700629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3269d5e-adb3-41d2-b20d-c3c692839b2b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:58.700941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3269d5e-adb3-41d2-b20d-c3c692839b2b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:58.701177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3269d5e-adb3-41d2-b20d-c3c692839b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:58.719423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166a75a-9e81-48dd-b65e-a98f14b69334 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:58.723098 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a3269d5e-adb3-41d2-b20d-c3c692839b2b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:58.724323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a3269d5e-adb3-41d2-b20d-c3c692839b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:58.761249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fc1b7-cf9c-4813-b476-179941de59a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:58.762244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fc1b7-cf9c-4813-b476-179941de59a7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:58.762760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fc1b7-cf9c-4813-b476-179941de59a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:58.763335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fc1b7-cf9c-4813-b476-179941de59a7 None admin] PATH_INFO: `/v3/users/5993366070364e2ea23f57f39e958895` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:58.765498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fc1b7-cf9c-4813-b476-179941de59a7 None admin] RBAC: Authorizing `identity:get_user(user_id=5993366070364e2ea23f57f39e958895)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:58.767763 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fc1b7-cf9c-4813-b476-179941de59a7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:58.769762 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 79/143] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:45:58 2022] GET /identity/v3/users/5993366070364e2ea23f57f39e958895 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:58.779971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fc1b7-cf9c-4813-b476-179941de59a7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:58.786262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:58.833487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 65/144] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:45:58.871406 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3269d5e-adb3-41d2-b20d-c3c692839b2b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:58.987264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:58.997073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fc1b7-cf9c-4813-b476-179941de59a7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:59.070101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3269d5e-adb3-41d2-b20d-c3c692839b2b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:59.107532 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5220600-7c6c-4677-9e0e-686638a10b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:59.108181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5220600-7c6c-4677-9e0e-686638a10b76 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:59.108473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5220600-7c6c-4677-9e0e-686638a10b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:59.108752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5220600-7c6c-4677-9e0e-686638a10b76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:59.140275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb22b7-5902-4ea3-bed3-adf3ae3cab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:59.140932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb22b7-5902-4ea3-bed3-adf3ae3cab5e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:59.141222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb22b7-5902-4ea3-bed3-adf3ae3cab5e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:59.141500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb22b7-5902-4ea3-bed3-adf3ae3cab5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:59.144189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb22b7-5902-4ea3-bed3-adf3ae3cab5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:59.145679 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb22b7-5902-4ea3-bed3-adf3ae3cab5e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:59.152726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 66/145] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:45:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:59.153970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb22b7-5902-4ea3-bed3-adf3ae3cab5e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:59.203528 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5220600-7c6c-4677-9e0e-686638a10b76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:59.205924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5220600-7c6c-4677-9e0e-686638a10b76 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:59.208183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:59.314879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 80/146] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:45:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:45:59.321388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5220600-7c6c-4677-9e0e-686638a10b76 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:59.439584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5220600-7c6c-4677-9e0e-686638a10b76 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:59.441343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:59.445479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb22b7-5902-4ea3-bed3-adf3ae3cab5e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:59.496084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edb071-b597-42ed-b9c3-00d0175c905e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:59.496750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edb071-b597-42ed-b9c3-00d0175c905e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:59.497035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edb071-b597-42ed-b9c3-00d0175c905e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:59.497313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edb071-b597-42ed-b9c3-00d0175c905e None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:59.506964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:59.507111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edb071-b597-42ed-b9c3-00d0175c905e None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:59.507111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edb071-b597-42ed-b9c3-00d0175c905e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:59.507637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 81/147] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:45:59 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:45:59.511200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86967fc-8fdc-45a8-aa8f-f45c344afa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:59.515528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86967fc-8fdc-45a8-aa8f-f45c344afa90 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:59.515776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edb071-b597-42ed-b9c3-00d0175c905e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:45:59.516162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86967fc-8fdc-45a8-aa8f-f45c344afa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:59.516604 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86967fc-8fdc-45a8-aa8f-f45c344afa90 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:59.517455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86967fc-8fdc-45a8-aa8f-f45c344afa90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:59.519094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86967fc-8fdc-45a8-aa8f-f45c344afa90 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:59.594382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 67/148] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:45:59 2022] POST /identity/v3/users => generated 283 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:45:59.750577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edb071-b597-42ed-b9c3-00d0175c905e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:45:59.814222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748740e-d8a3-4ebe-a529-2e2028993c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:45:59.814887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748740e-d8a3-4ebe-a529-2e2028993c20 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:59.815172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748740e-d8a3-4ebe-a529-2e2028993c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:59.815444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748740e-d8a3-4ebe-a529-2e2028993c20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:59.816261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748740e-d8a3-4ebe-a529-2e2028993c20 None admin] RBAC: Adding query filter params (role.id=f66cbbcc2491438184f04522c2a68578, scope.project.id=a75d68fb290c4a1494243d8817754029, user.id=5993366070364e2ea23f57f39e958895) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:45:59.816570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748740e-d8a3-4ebe-a529-2e2028993c20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:45:59.818058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748740e-d8a3-4ebe-a529-2e2028993c20 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:45:59.829481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 82/149] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:45:59 2022] GET /identity/v3/role_assignments?user.id=5993366070364e2ea23f57f39e958895&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=f66cbbcc2491438184f04522c2a68578 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:45:59.870137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ad85a-184b-4a40-9105-05afdf30c4ae None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:59.870324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ad85a-184b-4a40-9105-05afdf30c4ae None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:59.870615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ad85a-184b-4a40-9105-05afdf30c4ae None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:59.872092 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 68/150] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:45:59.877860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc32c84-03be-4f7a-a7eb-9c91120b12b6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:45:59.878146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc32c84-03be-4f7a-a7eb-9c91120b12b6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:45:59.878419 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc32c84-03be-4f7a-a7eb-9c91120b12b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:45:59.902251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7bc32c84-03be-4f7a-a7eb-9c91120b12b6 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:45:59.903507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7bc32c84-03be-4f7a-a7eb-9c91120b12b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:45:59.942998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:45:59.998218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 83/151] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:00.029339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ad85a-184b-4a40-9105-05afdf30c4ae None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:00.161991 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ad85a-184b-4a40-9105-05afdf30c4ae None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:00.200520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07272af4-928d-48c5-8444-7ca62d8033a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:00.201173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07272af4-928d-48c5-8444-7ca62d8033a6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:00.201458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07272af4-928d-48c5-8444-7ca62d8033a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:00.201732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07272af4-928d-48c5-8444-7ca62d8033a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:00.215425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07272af4-928d-48c5-8444-7ca62d8033a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-07272af4-928d-48c5-8444-7ca62d8033a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:00.220554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:00.221511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:00.222305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 69/152] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:46:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:00.228780 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc32c84-03be-4f7a-a7eb-9c91120b12b6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:00.320280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd372a-48c0-4ed8-80d6-41eb0159d357 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:00.321116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd372a-48c0-4ed8-80d6-41eb0159d357 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:00.321778 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd372a-48c0-4ed8-80d6-41eb0159d357 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:00.324086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 70/153] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:00.332234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646051ca-8afd-4d9a-a767-b3f56f39c0ef None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:00.332792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646051ca-8afd-4d9a-a767-b3f56f39c0ef None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:00.333376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646051ca-8afd-4d9a-a767-b3f56f39c0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:00.356383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc32c84-03be-4f7a-a7eb-9c91120b12b6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:00.362965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-646051ca-8afd-4d9a-a767-b3f56f39c0ef None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:00.364541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-646051ca-8afd-4d9a-a767-b3f56f39c0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:00.405297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14140fd-32a9-4350-96bf-eb687333b855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:00.405919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14140fd-32a9-4350-96bf-eb687333b855 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:00.406206 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14140fd-32a9-4350-96bf-eb687333b855 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:00.406486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14140fd-32a9-4350-96bf-eb687333b855 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:00.407251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14140fd-32a9-4350-96bf-eb687333b855 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:00.407562 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14140fd-32a9-4350-96bf-eb687333b855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:00.408973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14140fd-32a9-4350-96bf-eb687333b855 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:00.422697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 84/154] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:00.430117 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14140fd-32a9-4350-96bf-eb687333b855 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:00.432601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:00.466227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 71/155] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:00.473025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:00.496540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646051ca-8afd-4d9a-a767-b3f56f39c0ef None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:00.607168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14140fd-32a9-4350-96bf-eb687333b855 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:00.659285 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646051ca-8afd-4d9a-a767-b3f56f39c0ef None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:00.680949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09a7cf-c038-411a-85f9-880d3fdc8cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:00.682122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09a7cf-c038-411a-85f9-880d3fdc8cbd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:00.683292 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09a7cf-c038-411a-85f9-880d3fdc8cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:00.683931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09a7cf-c038-411a-85f9-880d3fdc8cbd None admin] PATH_INFO: `/v3/users/5993366070364e2ea23f57f39e958895` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:00.686744 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09a7cf-c038-411a-85f9-880d3fdc8cbd None admin] RBAC: Authorizing `identity:get_user(user_id=5993366070364e2ea23f57f39e958895)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:00.689680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09a7cf-c038-411a-85f9-880d3fdc8cbd None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:00.692322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 85/156] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:00 2022] GET /identity/v3/users/5993366070364e2ea23f57f39e958895 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:00.698979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09a7cf-c038-411a-85f9-880d3fdc8cbd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:00.736266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17319c2-df28-4186-a44d-df93ebff7414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:00.737252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17319c2-df28-4186-a44d-df93ebff7414 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:00.737976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17319c2-df28-4186-a44d-df93ebff7414 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:00.743386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17319c2-df28-4186-a44d-df93ebff7414 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:00.760742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17319c2-df28-4186-a44d-df93ebff7414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a17319c2-df28-4186-a44d-df93ebff7414 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:00.765627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:00.766714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:00.767670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 72/157] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:46:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:00.773147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17319c2-df28-4186-a44d-df93ebff7414 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:00.833870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:00.840691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:00.848367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09a7cf-c038-411a-85f9-880d3fdc8cbd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:00.889978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713f0a4-6827-4b08-8ced-bb398ee946d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:00.890906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713f0a4-6827-4b08-8ced-bb398ee946d6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:00.891340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713f0a4-6827-4b08-8ced-bb398ee946d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:00.891781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713f0a4-6827-4b08-8ced-bb398ee946d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:00.950865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17319c2-df28-4186-a44d-df93ebff7414 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:00.970804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713f0a4-6827-4b08-8ced-bb398ee946d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:00.972628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713f0a4-6827-4b08-8ced-bb398ee946d6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:01.006880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f89b0-02df-44df-bbc6-182c9ea4cd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:01.007808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f89b0-02df-44df-bbc6-182c9ea4cd5b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:01.007949 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f89b0-02df-44df-bbc6-182c9ea4cd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:01.008174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f89b0-02df-44df-bbc6-182c9ea4cd5b None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:01.008907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f89b0-02df-44df-bbc6-182c9ea4cd5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:01.009204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f89b0-02df-44df-bbc6-182c9ea4cd5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:01.010602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f89b0-02df-44df-bbc6-182c9ea4cd5b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:01.025532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 73/158] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:01.030728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f89b0-02df-44df-bbc6-182c9ea4cd5b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:01.079817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:01.088996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 86/159] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:01.094962 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713f0a4-6827-4b08-8ced-bb398ee946d6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:01.163427 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:01.167885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f89b0-02df-44df-bbc6-182c9ea4cd5b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:01.206057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f9b10-12f3-428b-ba6b-cceb067d58ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:01.207009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f9b10-12f3-428b-ba6b-cceb067d58ba None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:01.207235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f9b10-12f3-428b-ba6b-cceb067d58ba None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:01.207628 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f9b10-12f3-428b-ba6b-cceb067d58ba None admin] PATH_INFO: `/v3/users/fe2947ccbc264fdba860153b59471d41` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:01.208909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f9b10-12f3-428b-ba6b-cceb067d58ba None admin] RBAC: Authorizing `identity:get_user(user_id=fe2947ccbc264fdba860153b59471d41)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:01.211374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f9b10-12f3-428b-ba6b-cceb067d58ba None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:01.213231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 74/160] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:01 2022] GET /identity/v3/users/fe2947ccbc264fdba860153b59471d41 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:01.214602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b821df-37b9-4625-81a1-97f843bf19b0 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:01.215007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b821df-37b9-4625-81a1-97f843bf19b0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:01.215660 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b821df-37b9-4625-81a1-97f843bf19b0 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:01.216713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 75/161] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:01.219904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b821df-37b9-4625-81a1-97f843bf19b0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:01.229273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713f0a4-6827-4b08-8ced-bb398ee946d6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:01.268643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d628fd-85c2-4917-9cc9-f8bf94b098cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:01.269705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d628fd-85c2-4917-9cc9-f8bf94b098cd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:01.270250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d628fd-85c2-4917-9cc9-f8bf94b098cd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:01.270748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d628fd-85c2-4917-9cc9-f8bf94b098cd None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:01.273095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d628fd-85c2-4917-9cc9-f8bf94b098cd None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:01.275129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d628fd-85c2-4917-9cc9-f8bf94b098cd None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:01.277351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 87/162] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:01 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:01.278705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859a306-8e7e-4f01-b9bc-9df655e0ea7c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:01.279284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859a306-8e7e-4f01-b9bc-9df655e0ea7c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:01.279880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859a306-8e7e-4f01-b9bc-9df655e0ea7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:01.305089 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f859a306-8e7e-4f01-b9bc-9df655e0ea7c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:01.306490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f859a306-8e7e-4f01-b9bc-9df655e0ea7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:01.357363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b821df-37b9-4625-81a1-97f843bf19b0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:01.379065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:01.393489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 88/163] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:01.394620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859a306-8e7e-4f01-b9bc-9df655e0ea7c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:01.414629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd61be-bbb1-47a8-91c4-7e9578d76a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:01.415691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd61be-bbb1-47a8-91c4-7e9578d76a8e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:01.416218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd61be-bbb1-47a8-91c4-7e9578d76a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:01.416731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd61be-bbb1-47a8-91c4-7e9578d76a8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:01.463837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:01.514253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd61be-bbb1-47a8-91c4-7e9578d76a8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:01.515786 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd61be-bbb1-47a8-91c4-7e9578d76a8e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:01.601524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859a306-8e7e-4f01-b9bc-9df655e0ea7c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:01.668091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 76/164] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 03:46:01.669738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd61be-bbb1-47a8-91c4-7e9578d76a8e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:01.693258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0450bf-147b-4b09-a18b-3db333b68ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:01.695413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0450bf-147b-4b09-a18b-3db333b68ead None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:01.695725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0450bf-147b-4b09-a18b-3db333b68ead None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:01.696010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0450bf-147b-4b09-a18b-3db333b68ead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:01.696826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0450bf-147b-4b09-a18b-3db333b68ead None admin] RBAC: Adding query filter params (role.id=854a7f7c663b4c66a08f7e731c7bf63c, scope.project.id=a75d68fb290c4a1494243d8817754029, user.id=5993366070364e2ea23f57f39e958895) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:01.697144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0450bf-147b-4b09-a18b-3db333b68ead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:01.698632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0450bf-147b-4b09-a18b-3db333b68ead None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:01.716219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 89/165] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:01 2022] GET /identity/v3/role_assignments?user.id=5993366070364e2ea23f57f39e958895&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=854a7f7c663b4c66a08f7e731c7bf63c => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:01.717415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cf198-96a8-4082-b476-7608a75b2049 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:01.717701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cf198-96a8-4082-b476-7608a75b2049 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:01.717976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cf198-96a8-4082-b476-7608a75b2049 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:01.727586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 90/166] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:01 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:01.728489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cf198-96a8-4082-b476-7608a75b2049 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:01.784518 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:01.858954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd61be-bbb1-47a8-91c4-7e9578d76a8e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:01.935602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:01.937693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212ddd4-2ca7-4f5c-9c03-3e85b6ebe844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:01.938635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212ddd4-2ca7-4f5c-9c03-3e85b6ebe844 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:01.939267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212ddd4-2ca7-4f5c-9c03-3e85b6ebe844 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:01.939784 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212ddd4-2ca7-4f5c-9c03-3e85b6ebe844 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:01.966857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212ddd4-2ca7-4f5c-9c03-3e85b6ebe844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7212ddd4-2ca7-4f5c-9c03-3e85b6ebe844 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:01.972333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:01.973180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:01.974199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:01.976778 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 77/167] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:01.978189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083d7e-f88a-42d4-a8f3-6f059aee7731 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:01.978718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083d7e-f88a-42d4-a8f3-6f059aee7731 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:01.979261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083d7e-f88a-42d4-a8f3-6f059aee7731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:02.012958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fe083d7e-f88a-42d4-a8f3-6f059aee7731 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:02.014170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fe083d7e-f88a-42d4-a8f3-6f059aee7731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:02.044431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cf198-96a8-4082-b476-7608a75b2049 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:02.106134 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 78/168] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:02.107040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe083d7e-f88a-42d4-a8f3-6f059aee7731 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:02.144045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388af86-2872-4d0a-9501-8696e0950a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:02.144727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388af86-2872-4d0a-9501-8696e0950a1d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:02.144983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388af86-2872-4d0a-9501-8696e0950a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:02.145264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388af86-2872-4d0a-9501-8696e0950a1d None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:02.151514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388af86-2872-4d0a-9501-8696e0950a1d None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:02.152978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388af86-2872-4d0a-9501-8696e0950a1d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:02.154692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 91/169] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:01 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:02.161464 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388af86-2872-4d0a-9501-8696e0950a1d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:02.205232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:02.287353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe083d7e-f88a-42d4-a8f3-6f059aee7731 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:02.334171 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388af86-2872-4d0a-9501-8696e0950a1d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:02.338687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b448f-b27d-4015-a117-9af4a6de91ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:02.339360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b448f-b27d-4015-a117-9af4a6de91ef None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:02.339656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b448f-b27d-4015-a117-9af4a6de91ef None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:02.339930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b448f-b27d-4015-a117-9af4a6de91ef None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:02.340666 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b448f-b27d-4015-a117-9af4a6de91ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:02.340983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b448f-b27d-4015-a117-9af4a6de91ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:02.342356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b448f-b27d-4015-a117-9af4a6de91ef None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:02.359194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 79/170] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:02.360547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b448f-b27d-4015-a117-9af4a6de91ef None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:02.363858 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:02.387115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c81651-1c6a-40d9-8e74-d33e88025e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:02.387741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c81651-1c6a-40d9-8e74-d33e88025e46 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:02.388025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c81651-1c6a-40d9-8e74-d33e88025e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:02.388413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c81651-1c6a-40d9-8e74-d33e88025e46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:02.389237 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c81651-1c6a-40d9-8e74-d33e88025e46 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=a75d68fb290c4a1494243d8817754029, user.id=fe2947ccbc264fdba860153b59471d41) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:02.389550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c81651-1c6a-40d9-8e74-d33e88025e46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:02.391931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c81651-1c6a-40d9-8e74-d33e88025e46 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:02.404819 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 92/171] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:02 2022] GET /identity/v3/role_assignments?user.id=fe2947ccbc264fdba860153b59471d41&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=a37ae24c516b48c69e13721c013134a2 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:02.406432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c81651-1c6a-40d9-8e74-d33e88025e46 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:02.561846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c81651-1c6a-40d9-8e74-d33e88025e46 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:02.607299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b448f-b27d-4015-a117-9af4a6de91ef None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:02.618604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcf863-a6bc-4113-b8eb-ceff08cd687b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:02.619361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcf863-a6bc-4113-b8eb-ceff08cd687b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:02.619696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcf863-a6bc-4113-b8eb-ceff08cd687b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:02.619983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcf863-a6bc-4113-b8eb-ceff08cd687b None admin] PATH_INFO: `/v3/users/5993366070364e2ea23f57f39e958895` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:02.621225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcf863-a6bc-4113-b8eb-ceff08cd687b None admin] RBAC: Authorizing `identity:get_user(user_id=5993366070364e2ea23f57f39e958895)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:02.622681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcf863-a6bc-4113-b8eb-ceff08cd687b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:02.624343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 93/172] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:02 2022] GET /identity/v3/users/5993366070364e2ea23f57f39e958895 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:02.631713 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcf863-a6bc-4113-b8eb-ceff08cd687b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:02.660198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:02.703724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33232803-bc49-4746-83db-4142776e79dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:02.704431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33232803-bc49-4746-83db-4142776e79dd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:02.704719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33232803-bc49-4746-83db-4142776e79dd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:02.705088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33232803-bc49-4746-83db-4142776e79dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:02.739380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33232803-bc49-4746-83db-4142776e79dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-33232803-bc49-4746-83db-4142776e79dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:02.745334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:02.747420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:02.749425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:02.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 80/173] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:02.755802 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33232803-bc49-4746-83db-4142776e79dd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:02.769560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:02.869015 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcf863-a6bc-4113-b8eb-ceff08cd687b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:02.948981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ecb6e-827f-42e3-9ccf-8232ae98b63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:02.949670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ecb6e-827f-42e3-9ccf-8232ae98b63d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:02.949957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ecb6e-827f-42e3-9ccf-8232ae98b63d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:02.950235 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ecb6e-827f-42e3-9ccf-8232ae98b63d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:02.954509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33232803-bc49-4746-83db-4142776e79dd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:03.002001 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1dd13-c5fc-4f51-b251-f0628e09c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:03.002643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1dd13-c5fc-4f51-b251-f0628e09c331 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.002967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1dd13-c5fc-4f51-b251-f0628e09c331 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.003246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1dd13-c5fc-4f51-b251-f0628e09c331 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.003984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1dd13-c5fc-4f51-b251-f0628e09c331 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:03.004301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1dd13-c5fc-4f51-b251-f0628e09c331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:03.005690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1dd13-c5fc-4f51-b251-f0628e09c331 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:03.023689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 81/174] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:03.034856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1dd13-c5fc-4f51-b251-f0628e09c331 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:03.081783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ecb6e-827f-42e3-9ccf-8232ae98b63d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:03.082828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:03.085915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ecb6e-827f-42e3-9ccf-8232ae98b63d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:03.148580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1dd13-c5fc-4f51-b251-f0628e09c331 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:03.161144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:03.186762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284e21f-bc81-459f-a433-876701c434aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:03.187551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284e21f-bc81-459f-a433-876701c434aa None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.187906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284e21f-bc81-459f-a433-876701c434aa None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.188260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284e21f-bc81-459f-a433-876701c434aa None admin] PATH_INFO: `/v3/users/76cc90df40644f83ba6c7346544ddfec` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.189645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284e21f-bc81-459f-a433-876701c434aa None admin] RBAC: Authorizing `identity:get_user(user_id=76cc90df40644f83ba6c7346544ddfec)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:03.191305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284e21f-bc81-459f-a433-876701c434aa None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:03.193198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 82/175] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:03 2022] GET /identity/v3/users/76cc90df40644f83ba6c7346544ddfec => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:03.199567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08aa6c1-5761-4506-9c6f-b4b0272dfa78 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.199868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08aa6c1-5761-4506-9c6f-b4b0272dfa78 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.200152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08aa6c1-5761-4506-9c6f-b4b0272dfa78 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.201444 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 83/176] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:03.202157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08aa6c1-5761-4506-9c6f-b4b0272dfa78 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:03.219643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 94/177] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:03.221655 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d9e7c-2419-4c3c-894a-11697fb28f0f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.222400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d9e7c-2419-4c3c-894a-11697fb28f0f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.222907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d9e7c-2419-4c3c-894a-11697fb28f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.251232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fa5d9e7c-2419-4c3c-894a-11697fb28f0f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:03.251232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fa5d9e7c-2419-4c3c-894a-11697fb28f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:03.316817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08aa6c1-5761-4506-9c6f-b4b0272dfa78 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:03.330814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 95/178] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:03.330814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d9e7c-2419-4c3c-894a-11697fb28f0f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:03.334962 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:03.379994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ef40f-1d42-4c6d-a930-f2eec2d1e186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:03.380919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ef40f-1d42-4c6d-a930-f2eec2d1e186 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.381947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ef40f-1d42-4c6d-a930-f2eec2d1e186 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.382484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ef40f-1d42-4c6d-a930-f2eec2d1e186 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.383905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ef40f-1d42-4c6d-a930-f2eec2d1e186 None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:03.385559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ef40f-1d42-4c6d-a930-f2eec2d1e186 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:03.387543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 84/179] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:03 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:03.392974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ef40f-1d42-4c6d-a930-f2eec2d1e186 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:03.420063 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:03.438129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d9e7c-2419-4c3c-894a-11697fb28f0f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:03.475873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389ce62-a2fb-4d4e-b93e-5e6701480460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:03.476675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389ce62-a2fb-4d4e-b93e-5e6701480460 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.476959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389ce62-a2fb-4d4e-b93e-5e6701480460 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.477235 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389ce62-a2fb-4d4e-b93e-5e6701480460 None admin] PATH_INFO: `/v3/projects/d59be85cc0f8416f952e6e214a0aa96f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.478413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389ce62-a2fb-4d4e-b93e-5e6701480460 None admin] RBAC: Authorizing `identity:get_project(project_id=d59be85cc0f8416f952e6e214a0aa96f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:03.480075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389ce62-a2fb-4d4e-b93e-5e6701480460 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:03.482039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 96/180] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:03 2022] GET /identity/v3/projects/d59be85cc0f8416f952e6e214a0aa96f => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:03.482936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389ce62-a2fb-4d4e-b93e-5e6701480460 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:03.503826 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ef40f-1d42-4c6d-a930-f2eec2d1e186 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:03.541483 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb8144-6ff7-46ef-8102-117a8d971c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:03.542343 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb8144-6ff7-46ef-8102-117a8d971c5e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.542843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb8144-6ff7-46ef-8102-117a8d971c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.543297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb8144-6ff7-46ef-8102-117a8d971c5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.555374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb8144-6ff7-46ef-8102-117a8d971c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a7fb8144-6ff7-46ef-8102-117a8d971c5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:03.561202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:03.562073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:03.562909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:03.565271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 85/181] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:46:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:03.571312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb8144-6ff7-46ef-8102-117a8d971c5e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:03.597322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389ce62-a2fb-4d4e-b93e-5e6701480460 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:03.610595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:03.636095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a94fd6-887c-45dc-a586-afc5c000ec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:03.636842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a94fd6-887c-45dc-a586-afc5c000ec95 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.637057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a94fd6-887c-45dc-a586-afc5c000ec95 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.637330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a94fd6-887c-45dc-a586-afc5c000ec95 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029/users/76cc90df40644f83ba6c7346544ddfec/roles/77b5c5dec7334e27a8126a96b7b17f32` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.639354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a94fd6-887c-45dc-a586-afc5c000ec95 None admin] RBAC: Authorizing `identity:create_grant(project_id=a75d68fb290c4a1494243d8817754029, user_id=76cc90df40644f83ba6c7346544ddfec, role_id=77b5c5dec7334e27a8126a96b7b17f32)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:03.640875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a94fd6-887c-45dc-a586-afc5c000ec95 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:03.658844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 97/182] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:46:03 2022] PUT /identity/v3/projects/a75d68fb290c4a1494243d8817754029/users/76cc90df40644f83ba6c7346544ddfec/roles/77b5c5dec7334e27a8126a96b7b17f32 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:03.658844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a94fd6-887c-45dc-a586-afc5c000ec95 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:03.683161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:03.683957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb8144-6ff7-46ef-8102-117a8d971c5e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:03.724459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd3775-3d2a-45bd-a2d1-d0e8028147ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:03.725684 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd3775-3d2a-45bd-a2d1-d0e8028147ab None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.725994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd3775-3d2a-45bd-a2d1-d0e8028147ab None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.726279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd3775-3d2a-45bd-a2d1-d0e8028147ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.727138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd3775-3d2a-45bd-a2d1-d0e8028147ab None admin] RBAC: Adding query filter params (role.id=f66cbbcc2491438184f04522c2a68578, scope.project.id=d59be85cc0f8416f952e6e214a0aa96f, user.id=5993366070364e2ea23f57f39e958895) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:03.727455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd3775-3d2a-45bd-a2d1-d0e8028147ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:03.728978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd3775-3d2a-45bd-a2d1-d0e8028147ab None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:03.738375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 86/183] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:03 2022] GET /identity/v3/role_assignments?user.id=5993366070364e2ea23f57f39e958895&scope.project.id=d59be85cc0f8416f952e6e214a0aa96f&role.id=f66cbbcc2491438184f04522c2a68578 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:03.738375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a5c40-8d3a-475a-91fe-57084a654bd6 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.738375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a5c40-8d3a-475a-91fe-57084a654bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.738375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a5c40-8d3a-475a-91fe-57084a654bd6 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.738775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 87/184] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:03.740159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f36545-2fd9-4a85-90d7-11398a24dbee None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.740434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f36545-2fd9-4a85-90d7-11398a24dbee None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.742049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f36545-2fd9-4a85-90d7-11398a24dbee None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.742998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 88/185] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:03.745153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27bbc5-c182-4308-bd04-3fe2ed52e781 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.745506 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27bbc5-c182-4308-bd04-3fe2ed52e781 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.745786 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27bbc5-c182-4308-bd04-3fe2ed52e781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.767631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-af27bbc5-c182-4308-bd04-3fe2ed52e781 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:03.769037 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-af27bbc5-c182-4308-bd04-3fe2ed52e781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:03.811181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a94fd6-887c-45dc-a586-afc5c000ec95 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:03.857573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 89/186] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:03.859957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf4fb3-f48a-4117-87d7-8e2d34bcb29e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.860407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf4fb3-f48a-4117-87d7-8e2d34bcb29e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.860822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf4fb3-f48a-4117-87d7-8e2d34bcb29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.861300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ac26b-aff4-4252-b63f-8e724b233ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:03.861937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac26b-aff4-4252-b63f-8e724b233ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:03.862223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac26b-aff4-4252-b63f-8e724b233ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:03.862504 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac26b-aff4-4252-b63f-8e724b233ae0 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:03.863328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ac26b-aff4-4252-b63f-8e724b233ae0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:03.863641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ac26b-aff4-4252-b63f-8e724b233ae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:03.865175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ac26b-aff4-4252-b63f-8e724b233ae0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:03.875990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 98/187] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:03.882256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ac26b-aff4-4252-b63f-8e724b233ae0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:03.883570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-feaf4fb3-f48a-4117-87d7-8e2d34bcb29e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:03.884922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-feaf4fb3-f48a-4117-87d7-8e2d34bcb29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:03.919986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:03.980635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:03.982648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 90/188] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:03.983368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf4fb3-f48a-4117-87d7-8e2d34bcb29e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:04.020259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ac26b-aff4-4252-b63f-8e724b233ae0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:04.060800 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1ca43-5658-412b-9cd7-0b58c50ea6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:04.061446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1ca43-5658-412b-9cd7-0b58c50ea6af None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:04.061727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1ca43-5658-412b-9cd7-0b58c50ea6af None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:04.062001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1ca43-5658-412b-9cd7-0b58c50ea6af None admin] PATH_INFO: `/v3/users/13526544506140c5a5f223e306a2abd6` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:04.076707 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1ca43-5658-412b-9cd7-0b58c50ea6af None admin] RBAC: Authorizing `identity:get_user(user_id=13526544506140c5a5f223e306a2abd6)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:04.078178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1ca43-5658-412b-9cd7-0b58c50ea6af None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:04.084172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 99/189] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:03 2022] GET /identity/v3/users/13526544506140c5a5f223e306a2abd6 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:04.084172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1ca43-5658-412b-9cd7-0b58c50ea6af None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:04.105094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf4fb3-f48a-4117-87d7-8e2d34bcb29e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:04.141808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3843d-7aab-415e-9aee-ee0ef9c8c3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:04.142562 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3843d-7aab-415e-9aee-ee0ef9c8c3ed None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:04.142733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3843d-7aab-415e-9aee-ee0ef9c8c3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:04.143022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3843d-7aab-415e-9aee-ee0ef9c8c3ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:04.154226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3843d-7aab-415e-9aee-ee0ef9c8c3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-24d3843d-7aab-415e-9aee-ee0ef9c8c3ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:04.159340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:04.160198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:04.160980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 91/190] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:46:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:04.161885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3843d-7aab-415e-9aee-ee0ef9c8c3ed None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:04.190837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:04.196651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1ca43-5658-412b-9cd7-0b58c50ea6af None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:04.236359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9d667-6c03-4927-bb36-71662ea97ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:04.236930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9d667-6c03-4927-bb36-71662ea97ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:04.237215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9d667-6c03-4927-bb36-71662ea97ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:04.237493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9d667-6c03-4927-bb36-71662ea97ed1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:04.238652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9d667-6c03-4927-bb36-71662ea97ed1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:04.240135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9d667-6c03-4927-bb36-71662ea97ed1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:04.242528 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 100/191] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:04.243106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9d667-6c03-4927-bb36-71662ea97ed1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:04.249116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:04.287078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3843d-7aab-415e-9aee-ee0ef9c8c3ed None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:04.328733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447e25e-bd26-4e3b-bb51-30cb3b2a73f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:04.329356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447e25e-bd26-4e3b-bb51-30cb3b2a73f9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:04.329751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447e25e-bd26-4e3b-bb51-30cb3b2a73f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:04.330090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447e25e-bd26-4e3b-bb51-30cb3b2a73f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:04.351559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9d667-6c03-4927-bb36-71662ea97ed1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:04.388372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1857a-266a-4f0a-bea7-ea341ecdd885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:04.395666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1857a-266a-4f0a-bea7-ea341ecdd885 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:04.395666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1857a-266a-4f0a-bea7-ea341ecdd885 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:04.395666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1857a-266a-4f0a-bea7-ea341ecdd885 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:04.395666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1857a-266a-4f0a-bea7-ea341ecdd885 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:04.395666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1857a-266a-4f0a-bea7-ea341ecdd885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:04.395666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1857a-266a-4f0a-bea7-ea341ecdd885 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:04.408361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 101/192] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:04.408361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1857a-266a-4f0a-bea7-ea341ecdd885 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:04.413738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447e25e-bd26-4e3b-bb51-30cb3b2a73f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:04.415281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447e25e-bd26-4e3b-bb51-30cb3b2a73f9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:04.436943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:04.477672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:04.520577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1857a-266a-4f0a-bea7-ea341ecdd885 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:04.521452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 92/193] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:04.526754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447e25e-bd26-4e3b-bb51-30cb3b2a73f9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:04.564167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a080f1a-ae82-4433-8616-d22c54b74c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:04.564801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a080f1a-ae82-4433-8616-d22c54b74c6e None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:04.565088 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a080f1a-ae82-4433-8616-d22c54b74c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:04.565364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a080f1a-ae82-4433-8616-d22c54b74c6e None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:04.566172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a080f1a-ae82-4433-8616-d22c54b74c6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:04.567776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a080f1a-ae82-4433-8616-d22c54b74c6e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:04.602821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 102/194] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:46:04 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:46:04.602821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a080f1a-ae82-4433-8616-d22c54b74c6e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:04.644815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447e25e-bd26-4e3b-bb51-30cb3b2a73f9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:04.669907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:04.695653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d576e2-2391-465d-8ffd-0de0f4e7bac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:04.696323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d576e2-2391-465d-8ffd-0de0f4e7bac2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:04.696605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d576e2-2391-465d-8ffd-0de0f4e7bac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:04.696878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d576e2-2391-465d-8ffd-0de0f4e7bac2 None admin] PATH_INFO: `/v3/users/13526544506140c5a5f223e306a2abd6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:04.699026 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d576e2-2391-465d-8ffd-0de0f4e7bac2 None admin] RBAC: Authorizing `identity:get_user(user_id=13526544506140c5a5f223e306a2abd6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:04.701327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d576e2-2391-465d-8ffd-0de0f4e7bac2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:04.704737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 93/195] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:04 2022] GET /identity/v3/users/13526544506140c5a5f223e306a2abd6 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:04.711709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d576e2-2391-465d-8ffd-0de0f4e7bac2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:04.750300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a080f1a-ae82-4433-8616-d22c54b74c6e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:04.776217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:04.799617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ccf5d-6511-4031-8811-6e0ee8d464c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:04.800285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ccf5d-6511-4031-8811-6e0ee8d464c2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:04.800573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ccf5d-6511-4031-8811-6e0ee8d464c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:04.800853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ccf5d-6511-4031-8811-6e0ee8d464c2 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:04.802222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ccf5d-6511-4031-8811-6e0ee8d464c2 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:04.805354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ccf5d-6511-4031-8811-6e0ee8d464c2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:04.808338 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 103/196] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:04 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:04.814362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ccf5d-6511-4031-8811-6e0ee8d464c2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:04.848112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d576e2-2391-465d-8ffd-0de0f4e7bac2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:04.888300 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8365fec-bb2b-4505-a621-bc6e4e30a83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:04.889389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8365fec-bb2b-4505-a621-bc6e4e30a83a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:04.889878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8365fec-bb2b-4505-a621-bc6e4e30a83a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:04.890358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8365fec-bb2b-4505-a621-bc6e4e30a83a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:04.954914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ccf5d-6511-4031-8811-6e0ee8d464c2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:04.964365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:04.982535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8365fec-bb2b-4505-a621-bc6e4e30a83a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:04.984496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8365fec-bb2b-4505-a621-bc6e4e30a83a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:05.001206 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c890d-6f51-47d6-92ab-e7917930c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:05.002083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c890d-6f51-47d6-92ab-e7917930c616 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:05.002380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c890d-6f51-47d6-92ab-e7917930c616 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:05.002664 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c890d-6f51-47d6-92ab-e7917930c616 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:05.004261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c890d-6f51-47d6-92ab-e7917930c616 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=6409fa7c592e4bd09561cddf768267d4, user.id=13526544506140c5a5f223e306a2abd6) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:05.004383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c890d-6f51-47d6-92ab-e7917930c616 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:05.006077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c890d-6f51-47d6-92ab-e7917930c616 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:05.018817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 104/197] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:04 2022] GET /identity/v3/role_assignments?user.id=13526544506140c5a5f223e306a2abd6&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=a37ae24c516b48c69e13721c013134a2 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:05.108434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 94/198] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:05.116658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c890d-6f51-47d6-92ab-e7917930c616 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:05.207581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:05.265731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c890d-6f51-47d6-92ab-e7917930c616 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:05.322331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46770fe2-f6fa-4c73-b011-b0183275b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:05.322999 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46770fe2-f6fa-4c73-b011-b0183275b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:05.323294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46770fe2-f6fa-4c73-b011-b0183275b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:05.323557 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46770fe2-f6fa-4c73-b011-b0183275b4d2 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:05.324744 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46770fe2-f6fa-4c73-b011-b0183275b4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:05.326194 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46770fe2-f6fa-4c73-b011-b0183275b4d2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:05.327890 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 105/199] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:05 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:05.333788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8365fec-bb2b-4505-a621-bc6e4e30a83a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:05.464287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:05.478575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8365fec-bb2b-4505-a621-bc6e4e30a83a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:05.539549 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a55b45-fbcb-4822-90cf-d3714b01c48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:05.540186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a55b45-fbcb-4822-90cf-d3714b01c48f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:05.540957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a55b45-fbcb-4822-90cf-d3714b01c48f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:05.540957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a55b45-fbcb-4822-90cf-d3714b01c48f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:05.541505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a55b45-fbcb-4822-90cf-d3714b01c48f None admin] RBAC: Adding query filter params (role.id=854a7f7c663b4c66a08f7e731c7bf63c, scope.project.id=6409fa7c592e4bd09561cddf768267d4, user.id=13526544506140c5a5f223e306a2abd6) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:05.541811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a55b45-fbcb-4822-90cf-d3714b01c48f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:05.543291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a55b45-fbcb-4822-90cf-d3714b01c48f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:05.552998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 95/200] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:05 2022] GET /identity/v3/role_assignments?user.id=13526544506140c5a5f223e306a2abd6&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=854a7f7c663b4c66a08f7e731c7bf63c => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:05.919993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b9f33-2286-48e4-bd1a-54ef5f475086 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:05.920506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b9f33-2286-48e4-bd1a-54ef5f475086 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:05.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b9f33-2286-48e4-bd1a-54ef5f475086 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:05.922150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 106/201] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:05.926967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c944f-71af-48b6-b88c-1c7fe19daa8d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:05.927368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c944f-71af-48b6-b88c-1c7fe19daa8d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:05.927551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c944f-71af-48b6-b88c-1c7fe19daa8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:05.951537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fd5c944f-71af-48b6-b88c-1c7fe19daa8d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:05.952725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fd5c944f-71af-48b6-b88c-1c7fe19daa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:06.056842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 96/202] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:06.091488 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b9f33-2286-48e4-bd1a-54ef5f475086 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:06.259610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b9f33-2286-48e4-bd1a-54ef5f475086 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:06.281884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:06.303258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5feeaec-344f-4e1b-b684-42690116a405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:06.303914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5feeaec-344f-4e1b-b684-42690116a405 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:06.304202 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5feeaec-344f-4e1b-b684-42690116a405 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:06.304478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5feeaec-344f-4e1b-b684-42690116a405 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:06.328464 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5feeaec-344f-4e1b-b684-42690116a405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d5feeaec-344f-4e1b-b684-42690116a405 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:06.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:06.335764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:06.337203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 107/203] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:06.341629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c944f-71af-48b6-b88c-1c7fe19daa8d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:06.474101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:06.506751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c944f-71af-48b6-b88c-1c7fe19daa8d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:06.561716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d7035-66a3-4546-9090-4bc11723d404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:06.562375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d7035-66a3-4546-9090-4bc11723d404 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:06.562659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d7035-66a3-4546-9090-4bc11723d404 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:06.562971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d7035-66a3-4546-9090-4bc11723d404 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:06.564423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d7035-66a3-4546-9090-4bc11723d404 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:06.564926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d7035-66a3-4546-9090-4bc11723d404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:06.566352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d7035-66a3-4546-9090-4bc11723d404 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:06.576227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 97/204] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:06.582084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5feeaec-344f-4e1b-b684-42690116a405 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:06.716066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5feeaec-344f-4e1b-b684-42690116a405 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:06.763670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79564911-a75b-402a-8b4c-b7fc13e91146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:06.764749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79564911-a75b-402a-8b4c-b7fc13e91146 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:06.765255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79564911-a75b-402a-8b4c-b7fc13e91146 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:06.765745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79564911-a75b-402a-8b4c-b7fc13e91146 None admin] PATH_INFO: `/v3/users/76cc90df40644f83ba6c7346544ddfec` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:06.767455 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79564911-a75b-402a-8b4c-b7fc13e91146 None admin] RBAC: Authorizing `identity:get_user(user_id=76cc90df40644f83ba6c7346544ddfec)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:06.769169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79564911-a75b-402a-8b4c-b7fc13e91146 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:06.771416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 108/205] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:06 2022] GET /identity/v3/users/76cc90df40644f83ba6c7346544ddfec => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:06.777305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d7035-66a3-4546-9090-4bc11723d404 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:06.960192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d7035-66a3-4546-9090-4bc11723d404 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:07.010405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdb2bb-1b2e-4ca4-ace6-408a8dbe88b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:07.011067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdb2bb-1b2e-4ca4-ace6-408a8dbe88b6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:07.011354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdb2bb-1b2e-4ca4-ace6-408a8dbe88b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:07.011640 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdb2bb-1b2e-4ca4-ace6-408a8dbe88b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:07.027641 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:07.095237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdb2bb-1b2e-4ca4-ace6-408a8dbe88b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:07.096720 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdb2bb-1b2e-4ca4-ace6-408a8dbe88b6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:07.206293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 98/206] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:07.212618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79564911-a75b-402a-8b4c-b7fc13e91146 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:07.277500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:07.281441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98916c-2b00-45a4-999e-e6cdf3994275 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:07.281792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98916c-2b00-45a4-999e-e6cdf3994275 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:07.282071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98916c-2b00-45a4-999e-e6cdf3994275 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:07.283524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 99/207] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:07.290355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0726293-b2fd-487f-a233-5da08fe69c7a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:07.291103 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0726293-b2fd-487f-a233-5da08fe69c7a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:07.291958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0726293-b2fd-487f-a233-5da08fe69c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:07.318769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b0726293-b2fd-487f-a233-5da08fe69c7a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:07.320069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b0726293-b2fd-487f-a233-5da08fe69c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:07.349224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79564911-a75b-402a-8b4c-b7fc13e91146 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:07.392223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383b982-68ff-4f75-ac49-3496127abd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:07.392875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383b982-68ff-4f75-ac49-3496127abd28 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:07.393159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383b982-68ff-4f75-ac49-3496127abd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:07.393437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383b982-68ff-4f75-ac49-3496127abd28 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:07.396010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383b982-68ff-4f75-ac49-3496127abd28 None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:07.398395 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383b982-68ff-4f75-ac49-3496127abd28 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:07.403519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 109/208] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:07 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:07.404792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4be97-1743-4df7-bd6b-6437b54842bb None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:07.405084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4be97-1743-4df7-bd6b-6437b54842bb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:07.405368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4be97-1743-4df7-bd6b-6437b54842bb None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:07.410890 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 110/209] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:07.415653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1451-0e23-4742-80e2-3a8e28934a70 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:07.418808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1451-0e23-4742-80e2-3a8e28934a70 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:07.418808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1451-0e23-4742-80e2-3a8e28934a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:07.419692 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:07.451976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9bcd1451-0e23-4742-80e2-3a8e28934a70 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:07.453207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9bcd1451-0e23-4742-80e2-3a8e28934a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:07.471318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 100/210] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:07.472575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0726293-b2fd-487f-a233-5da08fe69c7a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:07.601133 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 111/211] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:07.602321 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd1451-0e23-4742-80e2-3a8e28934a70 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:07.677521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0726293-b2fd-487f-a233-5da08fe69c7a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:07.695113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:07.749033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767a24d-3ab8-4085-8ed2-b056ae060428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:07.750096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767a24d-3ab8-4085-8ed2-b056ae060428 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:07.750576 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767a24d-3ab8-4085-8ed2-b056ae060428 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:07.751103 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767a24d-3ab8-4085-8ed2-b056ae060428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:07.752187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767a24d-3ab8-4085-8ed2-b056ae060428 None admin] RBAC: Adding query filter params (role.id=77b5c5dec7334e27a8126a96b7b17f32, scope.project.id=a75d68fb290c4a1494243d8817754029, user.id=76cc90df40644f83ba6c7346544ddfec) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:07.752759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767a24d-3ab8-4085-8ed2-b056ae060428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:07.754447 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767a24d-3ab8-4085-8ed2-b056ae060428 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:07.761937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 101/212] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:07 2022] GET /identity/v3/role_assignments?user.id=76cc90df40644f83ba6c7346544ddfec&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=77b5c5dec7334e27a8126a96b7b17f32 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:07.763054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767a24d-3ab8-4085-8ed2-b056ae060428 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:07.784537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd1451-0e23-4742-80e2-3a8e28934a70 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:07.851379 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:07.872276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c145a-0cb9-4620-b443-4696cb861921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:07.873362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c145a-0cb9-4620-b443-4696cb861921 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:07.873860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c145a-0cb9-4620-b443-4696cb861921 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:07.874370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c145a-0cb9-4620-b443-4696cb861921 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:07.898604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c145a-0cb9-4620-b443-4696cb861921 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-ce3c145a-0cb9-4620-b443-4696cb861921 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:07.904320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:07.905264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:07.906060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:07.915513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 112/213] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:07.924525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c145a-0cb9-4620-b443-4696cb861921 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:07.992904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767a24d-3ab8-4085-8ed2-b056ae060428 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:08.046591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5534e-be0c-4619-8012-5506da1b42ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:08.047285 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5534e-be0c-4619-8012-5506da1b42ab None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.047572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5534e-be0c-4619-8012-5506da1b42ab None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.047852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5534e-be0c-4619-8012-5506da1b42ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.049024 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5534e-be0c-4619-8012-5506da1b42ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:08.050489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5534e-be0c-4619-8012-5506da1b42ab None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:08.052719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 102/214] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:08.054864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eea33c-60c0-41d5-9a1d-9ac630dbb3f5 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.054864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eea33c-60c0-41d5-9a1d-9ac630dbb3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.055054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eea33c-60c0-41d5-9a1d-9ac630dbb3f5 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.056306 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 103/215] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:08.058161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eea33c-60c0-41d5-9a1d-9ac630dbb3f5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:08.117209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c145a-0cb9-4620-b443-4696cb861921 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:08.126927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:08.157331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee1291-ee2a-4fbe-8f2c-21cbb9b1895f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:08.157956 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee1291-ee2a-4fbe-8f2c-21cbb9b1895f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.158236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee1291-ee2a-4fbe-8f2c-21cbb9b1895f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.158616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee1291-ee2a-4fbe-8f2c-21cbb9b1895f None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.159480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee1291-ee2a-4fbe-8f2c-21cbb9b1895f None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:08.159781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee1291-ee2a-4fbe-8f2c-21cbb9b1895f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:08.161182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee1291-ee2a-4fbe-8f2c-21cbb9b1895f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:08.171077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 113/216] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:08.172240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f133993-4473-474c-ac3d-2ef322b5af06 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.172528 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f133993-4473-474c-ac3d-2ef322b5af06 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.172798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f133993-4473-474c-ac3d-2ef322b5af06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.184891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eea33c-60c0-41d5-9a1d-9ac630dbb3f5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:08.191757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9f133993-4473-474c-ac3d-2ef322b5af06 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:08.192078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:08.193544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9f133993-4473-474c-ac3d-2ef322b5af06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:08.221156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c461a3-5f4e-46ea-9a73-059afc4d7eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:08.221774 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c461a3-5f4e-46ea-9a73-059afc4d7eb5 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.222057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c461a3-5f4e-46ea-9a73-059afc4d7eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.222332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c461a3-5f4e-46ea-9a73-059afc4d7eb5 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.223053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c461a3-5f4e-46ea-9a73-059afc4d7eb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:08.224505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c461a3-5f4e-46ea-9a73-059afc4d7eb5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:08.245900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 104/217] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:46:08 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:46:08.247041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c461a3-5f4e-46ea-9a73-059afc4d7eb5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:08.267811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 114/218] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:08.277061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efcc06-17b4-476b-862b-953d3a82f4e7 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.277819 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efcc06-17b4-476b-862b-953d3a82f4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.278376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efcc06-17b4-476b-862b-953d3a82f4e7 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.280270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 115/219] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:08 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:08.290238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdb88b-fa68-4513-be28-842804dfe4ba None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.290238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdb88b-fa68-4513-be28-842804dfe4ba None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.290238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdb88b-fa68-4513-be28-842804dfe4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.306370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6cbdb88b-fa68-4513-be28-842804dfe4ba None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:08.307669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6cbdb88b-fa68-4513-be28-842804dfe4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:08.373129 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c461a3-5f4e-46ea-9a73-059afc4d7eb5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:08.384243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 116/220] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:08.387711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdb88b-fa68-4513-be28-842804dfe4ba None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:08.423112 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:08.425010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6ef7b-dcdb-4c31-9bb7-c7b85a6b716a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:08.425293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6ef7b-dcdb-4c31-9bb7-c7b85a6b716a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.425583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6ef7b-dcdb-4c31-9bb7-c7b85a6b716a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.425855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6ef7b-dcdb-4c31-9bb7-c7b85a6b716a None admin] PATH_INFO: `/v3/users/ea7e2957d3b343448d008d2feb55ab6d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.440438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6ef7b-dcdb-4c31-9bb7-c7b85a6b716a None admin] RBAC: Authorizing `identity:get_user(user_id=ea7e2957d3b343448d008d2feb55ab6d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:08.441941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6ef7b-dcdb-4c31-9bb7-c7b85a6b716a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:08.444371 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 105/221] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:08 2022] GET /identity/v3/users/ea7e2957d3b343448d008d2feb55ab6d => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:08.445115 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6ef7b-dcdb-4c31-9bb7-c7b85a6b716a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:08.471078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:08.519201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdb88b-fa68-4513-be28-842804dfe4ba None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:08.566283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996baa1-c384-43f1-bddd-4528e95bd8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:08.566883 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996baa1-c384-43f1-bddd-4528e95bd8fd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.567170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996baa1-c384-43f1-bddd-4528e95bd8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.567447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996baa1-c384-43f1-bddd-4528e95bd8fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.580340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996baa1-c384-43f1-bddd-4528e95bd8fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:08.585951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6ef7b-dcdb-4c31-9bb7-c7b85a6b716a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1996baa1-c384-43f1-bddd-4528e95bd8fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:08.586087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:08.586988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:08.587783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 117/222] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:46:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:08.588449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996baa1-c384-43f1-bddd-4528e95bd8fd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:08.635568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c232ce-d733-4a39-bdce-bfdfdc2178b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:08.636828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c232ce-d733-4a39-bdce-bfdfdc2178b4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.638328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c232ce-d733-4a39-bdce-bfdfdc2178b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.638910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c232ce-d733-4a39-bdce-bfdfdc2178b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.654176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c232ce-d733-4a39-bdce-bfdfdc2178b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-24c232ce-d733-4a39-bdce-bfdfdc2178b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:08.660862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:08.662190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:08.663869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:08.666641 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 106/223] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:46:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:08.667754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c232ce-d733-4a39-bdce-bfdfdc2178b4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:08.716815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:08.736032 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996baa1-c384-43f1-bddd-4528e95bd8fd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:08.761028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:08.778143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207973-6489-45b4-adb5-eb9a73d3be22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:08.778880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207973-6489-45b4-adb5-eb9a73d3be22 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.779278 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207973-6489-45b4-adb5-eb9a73d3be22 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.779643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207973-6489-45b4-adb5-eb9a73d3be22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.789087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c232ce-d733-4a39-bdce-bfdfdc2178b4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:08.835916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc845c-e2e1-4575-83ae-39ff10d62860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:08.837018 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc845c-e2e1-4575-83ae-39ff10d62860 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:08.837828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc845c-e2e1-4575-83ae-39ff10d62860 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:08.838255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc845c-e2e1-4575-83ae-39ff10d62860 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:08.839169 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc845c-e2e1-4575-83ae-39ff10d62860 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:08.839634 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc845c-e2e1-4575-83ae-39ff10d62860 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:08.841719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc845c-e2e1-4575-83ae-39ff10d62860 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:08.853701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 107/224] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:08.855013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc845c-e2e1-4575-83ae-39ff10d62860 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:08.862370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89207973-6489-45b4-adb5-eb9a73d3be22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:08.863953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89207973-6489-45b4-adb5-eb9a73d3be22 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:08.980056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:08.985689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc845c-e2e1-4575-83ae-39ff10d62860 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:08.996674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 118/225] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:08.997719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207973-6489-45b4-adb5-eb9a73d3be22 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:09.028221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e218e-5eb1-4d88-b7aa-994c1304402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:09.028909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e218e-5eb1-4d88-b7aa-994c1304402e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:09.029221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e218e-5eb1-4d88-b7aa-994c1304402e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:09.029596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e218e-5eb1-4d88-b7aa-994c1304402e None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:09.030381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e218e-5eb1-4d88-b7aa-994c1304402e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:09.030702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e218e-5eb1-4d88-b7aa-994c1304402e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:09.032318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e218e-5eb1-4d88-b7aa-994c1304402e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:09.044059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 108/226] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:09.049123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e218e-5eb1-4d88-b7aa-994c1304402e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:09.053496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:09.125028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207973-6489-45b4-adb5-eb9a73d3be22 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:09.160655 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b4b79-36ca-4718-94a9-4bc482b0041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:09.161306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b4b79-36ca-4718-94a9-4bc482b0041c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:09.161888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b4b79-36ca-4718-94a9-4bc482b0041c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:09.162178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b4b79-36ca-4718-94a9-4bc482b0041c None admin] PATH_INFO: `/v3/users/13526544506140c5a5f223e306a2abd6` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:09.164327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b4b79-36ca-4718-94a9-4bc482b0041c None admin] RBAC: Authorizing `identity:get_user(user_id=13526544506140c5a5f223e306a2abd6)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:09.165999 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b4b79-36ca-4718-94a9-4bc482b0041c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:09.168457 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 119/227] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:08 2022] GET /identity/v3/users/13526544506140c5a5f223e306a2abd6 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:09.171645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b4b79-36ca-4718-94a9-4bc482b0041c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:09.186637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e218e-5eb1-4d88-b7aa-994c1304402e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:09.235592 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb39566-c3be-47fb-a08b-5086c4091c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:09.236501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb39566-c3be-47fb-a08b-5086c4091c9c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:09.236792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb39566-c3be-47fb-a08b-5086c4091c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:09.237074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb39566-c3be-47fb-a08b-5086c4091c9c None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:09.238069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:09.239958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb39566-c3be-47fb-a08b-5086c4091c9c None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:09.242440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb39566-c3be-47fb-a08b-5086c4091c9c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:09.244542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 109/228] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:09 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:09.246253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb39566-c3be-47fb-a08b-5086c4091c9c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:09.306208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b4b79-36ca-4718-94a9-4bc482b0041c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:09.325362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:09.348259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2f158-a319-4f22-8c9d-54fb39754b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:09.349213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2f158-a319-4f22-8c9d-54fb39754b2c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:09.349722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2f158-a319-4f22-8c9d-54fb39754b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:09.350248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2f158-a319-4f22-8c9d-54fb39754b2c None admin] PATH_INFO: `/v3/users/13526544506140c5a5f223e306a2abd6` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:09.351733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2f158-a319-4f22-8c9d-54fb39754b2c None admin] RBAC: Authorizing `identity:get_user(user_id=13526544506140c5a5f223e306a2abd6)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:09.353531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2f158-a319-4f22-8c9d-54fb39754b2c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:09.355702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 120/229] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:09 2022] GET /identity/v3/users/13526544506140c5a5f223e306a2abd6 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:09.356708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2f158-a319-4f22-8c9d-54fb39754b2c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:09.368684 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb39566-c3be-47fb-a08b-5086c4091c9c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:09.407022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3137e-67a2-4ab5-90f9-bd149edf1ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:09.407596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3137e-67a2-4ab5-90f9-bd149edf1ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:09.407895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3137e-67a2-4ab5-90f9-bd149edf1ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:09.408166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3137e-67a2-4ab5-90f9-bd149edf1ae1 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:09.409335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3137e-67a2-4ab5-90f9-bd149edf1ae1 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:09.410810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3137e-67a2-4ab5-90f9-bd149edf1ae1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:09.412959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 110/230] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:09 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:09.418345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3137e-67a2-4ab5-90f9-bd149edf1ae1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:09.470485 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2f158-a319-4f22-8c9d-54fb39754b2c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:09.498078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:09.510833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966bf71-f3ac-4c53-94cf-aa0a5b9f1515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:09.511756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966bf71-f3ac-4c53-94cf-aa0a5b9f1515 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:09.512242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966bf71-f3ac-4c53-94cf-aa0a5b9f1515 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:09.512781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966bf71-f3ac-4c53-94cf-aa0a5b9f1515 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:09.513829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966bf71-f3ac-4c53-94cf-aa0a5b9f1515 None admin] RBAC: Adding query filter params (role.id=f66cbbcc2491438184f04522c2a68578, scope.project.id=6409fa7c592e4bd09561cddf768267d4, user.id=ea7e2957d3b343448d008d2feb55ab6d) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:09.514359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966bf71-f3ac-4c53-94cf-aa0a5b9f1515 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:09.516145 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966bf71-f3ac-4c53-94cf-aa0a5b9f1515 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:09.523276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 121/231] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:09 2022] GET /identity/v3/role_assignments?user.id=ea7e2957d3b343448d008d2feb55ab6d&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=f66cbbcc2491438184f04522c2a68578 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:09.524388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966bf71-f3ac-4c53-94cf-aa0a5b9f1515 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:09.557094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3137e-67a2-4ab5-90f9-bd149edf1ae1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:09.646355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699efd2-cc30-456e-be56-60efd15fe7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:09.647472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699efd2-cc30-456e-be56-60efd15fe7cc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:09.648079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699efd2-cc30-456e-be56-60efd15fe7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:09.648557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699efd2-cc30-456e-be56-60efd15fe7cc None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:09.651548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699efd2-cc30-456e-be56-60efd15fe7cc None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:09.654006 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699efd2-cc30-456e-be56-60efd15fe7cc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:09.660630 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 111/232] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:09 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:09.664622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699efd2-cc30-456e-be56-60efd15fe7cc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:09.691039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:09.737191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966bf71-f3ac-4c53-94cf-aa0a5b9f1515 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:09.779509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dba11-cc93-4cba-b21c-b1aaa51ce48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:09.780296 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dba11-cc93-4cba-b21c-b1aaa51ce48d None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:09.780588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dba11-cc93-4cba-b21c-b1aaa51ce48d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:09.780870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dba11-cc93-4cba-b21c-b1aaa51ce48d None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4/users/13526544506140c5a5f223e306a2abd6/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:09.788856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dba11-cc93-4cba-b21c-b1aaa51ce48d None admin] RBAC: Authorizing `identity:create_grant(project_id=6409fa7c592e4bd09561cddf768267d4, user_id=13526544506140c5a5f223e306a2abd6, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:09.790314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dba11-cc93-4cba-b21c-b1aaa51ce48d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:09.792021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699efd2-cc30-456e-be56-60efd15fe7cc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:09.829643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd77dae-e766-4698-8e90-7db4a6188a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:09.829862 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 122/233] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:46:09 2022] PUT /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4/users/13526544506140c5a5f223e306a2abd6/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:09.830817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77dae-e766-4698-8e90-7db4a6188a16 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:09.831322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77dae-e766-4698-8e90-7db4a6188a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:09.838811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77dae-e766-4698-8e90-7db4a6188a16 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4/users/13526544506140c5a5f223e306a2abd6/roles/854a7f7c663b4c66a08f7e731c7bf63c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:09.841987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd77dae-e766-4698-8e90-7db4a6188a16 None admin] RBAC: Authorizing `identity:create_grant(project_id=6409fa7c592e4bd09561cddf768267d4, user_id=13526544506140c5a5f223e306a2abd6, role_id=854a7f7c663b4c66a08f7e731c7bf63c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:09.843711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd77dae-e766-4698-8e90-7db4a6188a16 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:09.869103 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 112/234] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:46:09 2022] PUT /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4/users/13526544506140c5a5f223e306a2abd6/roles/854a7f7c663b4c66a08f7e731c7bf63c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:10.429364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7765a4-f20e-47d5-a0e8-de69e98f0fed None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:10.429887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7765a4-f20e-47d5-a0e8-de69e98f0fed None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:10.430001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7765a4-f20e-47d5-a0e8-de69e98f0fed None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:10.431314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 123/235] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:10.436812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09f5d-befa-4b79-ae1f-f29e30acd584 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:10.437237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09f5d-befa-4b79-ae1f-f29e30acd584 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:10.437366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09f5d-befa-4b79-ae1f-f29e30acd584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:10.463855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8fe09f5d-befa-4b79-ae1f-f29e30acd584 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:10.465068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8fe09f5d-befa-4b79-ae1f-f29e30acd584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:10.540493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:10.552468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 113/236] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:10.577913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7765a4-f20e-47d5-a0e8-de69e98f0fed None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:10.618808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5230-299f-42a5-b5f0-0ca7edb70814 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:10.618808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5230-299f-42a5-b5f0-0ca7edb70814 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:10.618808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5230-299f-42a5-b5f0-0ca7edb70814 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:10.618808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 114/237] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:10.622711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2387-62fe-464e-873d-746179c6159f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:10.623042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2387-62fe-464e-873d-746179c6159f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:10.624052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2387-62fe-464e-873d-746179c6159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:10.639725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:10.661540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-516e2387-62fe-464e-873d-746179c6159f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:10.663198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-516e2387-62fe-464e-873d-746179c6159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:10.735092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7765a4-f20e-47d5-a0e8-de69e98f0fed None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:10.770004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 115/238] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:10.792398 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ce2a2-5f4e-4644-8481-a668a2b19bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:10.793047 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ce2a2-5f4e-4644-8481-a668a2b19bdb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:10.793331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ce2a2-5f4e-4644-8481-a668a2b19bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:10.793605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ce2a2-5f4e-4644-8481-a668a2b19bdb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:10.810324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ce2a2-5f4e-4644-8481-a668a2b19bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d49ce2a2-5f4e-4644-8481-a668a2b19bdb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:10.815694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:10.816547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:10.817356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 124/239] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:10.821442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e2387-62fe-464e-873d-746179c6159f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:10.824408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ce2a2-5f4e-4644-8481-a668a2b19bdb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:11.000184 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e2387-62fe-464e-873d-746179c6159f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:11.002264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ce2a2-5f4e-4644-8481-a668a2b19bdb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:11.018641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:11.051153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:11.062114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7cdac-8247-4d92-b689-edf4d92e84cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:11.062741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7cdac-8247-4d92-b689-edf4d92e84cc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:11.063048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7cdac-8247-4d92-b689-edf4d92e84cc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:11.063326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7cdac-8247-4d92-b689-edf4d92e84cc None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:11.064056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7cdac-8247-4d92-b689-edf4d92e84cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:11.064362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7cdac-8247-4d92-b689-edf4d92e84cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:11.065751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7cdac-8247-4d92-b689-edf4d92e84cc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:11.071462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be110c4a-a020-4e68-9f58-1d5f8c78d1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:11.072125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be110c4a-a020-4e68-9f58-1d5f8c78d1aa None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:11.072418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be110c4a-a020-4e68-9f58-1d5f8c78d1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:11.072701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be110c4a-a020-4e68-9f58-1d5f8c78d1aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:11.076220 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 125/240] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:11.080842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be110c4a-a020-4e68-9f58-1d5f8c78d1aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:11.080842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be110c4a-a020-4e68-9f58-1d5f8c78d1aa None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:11.081705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 116/241] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:11.082353 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7cdac-8247-4d92-b689-edf4d92e84cc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:11.087456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be110c4a-a020-4e68-9f58-1d5f8c78d1aa None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:11.257045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7cdac-8247-4d92-b689-edf4d92e84cc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:11.268074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be110c4a-a020-4e68-9f58-1d5f8c78d1aa None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:11.319480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739a0ca-eefd-4221-8097-bc8f9bcf3770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:11.321186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739a0ca-eefd-4221-8097-bc8f9bcf3770 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:11.322334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739a0ca-eefd-4221-8097-bc8f9bcf3770 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:11.323029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739a0ca-eefd-4221-8097-bc8f9bcf3770 None admin] PATH_INFO: `/v3/users/f5eafcfd910949638321d3a415669581` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:11.329885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6dff4-e88b-4040-b2fa-78b00467a7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:11.335493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6dff4-e88b-4040-b2fa-78b00467a7de None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:11.336051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6dff4-e88b-4040-b2fa-78b00467a7de None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:11.336527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6dff4-e88b-4040-b2fa-78b00467a7de None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:11.337441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6dff4-e88b-4040-b2fa-78b00467a7de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:11.339199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6dff4-e88b-4040-b2fa-78b00467a7de None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:11.354564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739a0ca-eefd-4221-8097-bc8f9bcf3770 None admin] RBAC: Authorizing `identity:get_user(user_id=f5eafcfd910949638321d3a415669581)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:11.356675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739a0ca-eefd-4221-8097-bc8f9bcf3770 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:11.364312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 126/242] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:11 2022] GET /identity/v3/users/f5eafcfd910949638321d3a415669581 => generated 297 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:11.367759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739a0ca-eefd-4221-8097-bc8f9bcf3770 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:11.388293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 117/243] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:46:11 2022] POST /identity/v3/users => generated 293 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:46:11.416801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:11.517476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739a0ca-eefd-4221-8097-bc8f9bcf3770 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:11.587078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a289ba-885c-41a8-a4f5-ccd7fdcae8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:11.587768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a289ba-885c-41a8-a4f5-ccd7fdcae8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:11.588053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a289ba-885c-41a8-a4f5-ccd7fdcae8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:11.588378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a289ba-885c-41a8-a4f5-ccd7fdcae8c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:11.658937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a289ba-885c-41a8-a4f5-ccd7fdcae8c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:11.660427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a289ba-885c-41a8-a4f5-ccd7fdcae8c8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:11.692544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:11.777824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 127/244] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:11.783358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6dff4-e88b-4040-b2fa-78b00467a7de None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:11.846977 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:11.880689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173ab62-8359-44ec-a080-a6cf06553121 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:11.881062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173ab62-8359-44ec-a080-a6cf06553121 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:11.881314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173ab62-8359-44ec-a080-a6cf06553121 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:11.882951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 128/245] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:46:11.888131 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a724907-26d0-4b12-ac63-a84871145aa6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:11.888417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a724907-26d0-4b12-ac63-a84871145aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:11.888697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a724907-26d0-4b12-ac63-a84871145aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:11.913876 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6dff4-e88b-4040-b2fa-78b00467a7de None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:11.916429 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6a724907-26d0-4b12-ac63-a84871145aa6 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:11.917257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6a724907-26d0-4b12-ac63-a84871145aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:11.973353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0267f78-ea15-4d0c-8018-63d545371793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:11.974016 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0267f78-ea15-4d0c-8018-63d545371793 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:11.974299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0267f78-ea15-4d0c-8018-63d545371793 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:11.974575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0267f78-ea15-4d0c-8018-63d545371793 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:11.976383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0267f78-ea15-4d0c-8018-63d545371793 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:11.978324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0267f78-ea15-4d0c-8018-63d545371793 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:11.981898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 118/246] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:11 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:11.986418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0267f78-ea15-4d0c-8018-63d545371793 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:12.025355 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 129/247] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:12.049355 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a724907-26d0-4b12-ac63-a84871145aa6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:12.111624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0267f78-ea15-4d0c-8018-63d545371793 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:12.116434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:12.133361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:12.150243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80300e7-a18d-498e-83e0-c13832ed71ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:12.150899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80300e7-a18d-498e-83e0-c13832ed71ff None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:12.151253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80300e7-a18d-498e-83e0-c13832ed71ff None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:12.151533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80300e7-a18d-498e-83e0-c13832ed71ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:12.152393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80300e7-a18d-498e-83e0-c13832ed71ff None admin] RBAC: Adding query filter params (role.id=77b5c5dec7334e27a8126a96b7b17f32, scope.project.id=6409fa7c592e4bd09561cddf768267d4, user.id=f5eafcfd910949638321d3a415669581) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:12.152714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80300e7-a18d-498e-83e0-c13832ed71ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:12.154179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80300e7-a18d-498e-83e0-c13832ed71ff None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:12.160752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 119/248] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:11 2022] GET /identity/v3/role_assignments?user.id=f5eafcfd910949638321d3a415669581&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=77b5c5dec7334e27a8126a96b7b17f32 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:12.164589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a724907-26d0-4b12-ac63-a84871145aa6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:12.203771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee31086-6844-4394-bc83-f53ae0012bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:12.204434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31086-6844-4394-bc83-f53ae0012bcb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:12.204715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31086-6844-4394-bc83-f53ae0012bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:12.204989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31086-6844-4394-bc83-f53ae0012bcb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:12.217510 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee31086-6844-4394-bc83-f53ae0012bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1ee31086-6844-4394-bc83-f53ae0012bcb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:12.222611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:12.223844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:12.225416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 130/249] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:12.232689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80300e7-a18d-498e-83e0-c13832ed71ff None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:12.387883 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80300e7-a18d-498e-83e0-c13832ed71ff None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:12.434309 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36c3a1-5098-458f-b423-35df3daa176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:12.434963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36c3a1-5098-458f-b423-35df3daa176b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:12.435246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36c3a1-5098-458f-b423-35df3daa176b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:12.435522 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36c3a1-5098-458f-b423-35df3daa176b None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:12.436256 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36c3a1-5098-458f-b423-35df3daa176b None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:12.436560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36c3a1-5098-458f-b423-35df3daa176b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:12.437934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36c3a1-5098-458f-b423-35df3daa176b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:12.447186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 120/250] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:12.452957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee31086-6844-4394-bc83-f53ae0012bcb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:12.562775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee31086-6844-4394-bc83-f53ae0012bcb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:12.581096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:12.604961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957300d-02f7-4135-8471-4a3e4b09edfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:12.605613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957300d-02f7-4135-8471-4a3e4b09edfa None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:12.605892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957300d-02f7-4135-8471-4a3e4b09edfa None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:12.606165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957300d-02f7-4135-8471-4a3e4b09edfa None admin] PATH_INFO: `/v3/users/ea7e2957d3b343448d008d2feb55ab6d` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:12.607722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957300d-02f7-4135-8471-4a3e4b09edfa None admin] RBAC: Authorizing `identity:get_user(user_id=ea7e2957d3b343448d008d2feb55ab6d)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:12.609172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957300d-02f7-4135-8471-4a3e4b09edfa None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:12.611199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 131/251] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:12 2022] GET /identity/v3/users/ea7e2957d3b343448d008d2feb55ab6d => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:12.613818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907e237-b223-43aa-948e-9001fda05116 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:12.614106 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907e237-b223-43aa-948e-9001fda05116 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:12.614378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907e237-b223-43aa-948e-9001fda05116 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:12.616516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 121/252] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:12.621156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957300d-02f7-4135-8471-4a3e4b09edfa None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:12.640637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a0a21-7ece-441b-a172-2b9c77812a4d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:12.641277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a0a21-7ece-441b-a172-2b9c77812a4d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:12.641759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a0a21-7ece-441b-a172-2b9c77812a4d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:12.648173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 122/253] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:12.649725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b1033-93df-41d1-9c0f-25587eca0cd2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:12.650216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b1033-93df-41d1-9c0f-25587eca0cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:12.650709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b1033-93df-41d1-9c0f-25587eca0cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:12.680582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-731b1033-93df-41d1-9c0f-25587eca0cd2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:12.682163 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-731b1033-93df-41d1-9c0f-25587eca0cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:12.775220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:12.778227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957300d-02f7-4135-8471-4a3e4b09edfa None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:12.828754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 123/254] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:12.829843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020314df-bf8f-4cff-9d8b-0b192eaed55f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:12.830131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020314df-bf8f-4cff-9d8b-0b192eaed55f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:12.830406 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020314df-bf8f-4cff-9d8b-0b192eaed55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:12.838631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29bd8c-f43e-4423-a236-6e47fd4b5304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:12.839314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29bd8c-f43e-4423-a236-6e47fd4b5304 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:12.839595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29bd8c-f43e-4423-a236-6e47fd4b5304 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:12.839871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29bd8c-f43e-4423-a236-6e47fd4b5304 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:12.842367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29bd8c-f43e-4423-a236-6e47fd4b5304 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:12.843846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29bd8c-f43e-4423-a236-6e47fd4b5304 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:12.845670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 132/255] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:12 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:12.855712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29bd8c-f43e-4423-a236-6e47fd4b5304 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:12.875833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-020314df-bf8f-4cff-9d8b-0b192eaed55f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:12.877034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-020314df-bf8f-4cff-9d8b-0b192eaed55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:12.923202 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:13.000083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29bd8c-f43e-4423-a236-6e47fd4b5304 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:13.010621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 124/256] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:13.011489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020314df-bf8f-4cff-9d8b-0b192eaed55f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:13.067987 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10023f-b36f-4a69-b4dc-4082c3a2f3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:13.070045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10023f-b36f-4a69-b4dc-4082c3a2f3a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:13.071178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10023f-b36f-4a69-b4dc-4082c3a2f3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:13.072173 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10023f-b36f-4a69-b4dc-4082c3a2f3a5 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4/users/ea7e2957d3b343448d008d2feb55ab6d/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:13.076341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10023f-b36f-4a69-b4dc-4082c3a2f3a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6409fa7c592e4bd09561cddf768267d4, user_id=ea7e2957d3b343448d008d2feb55ab6d, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:13.078665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10023f-b36f-4a69-b4dc-4082c3a2f3a5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:13.103397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 133/257] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:46:12 2022] PUT /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4/users/ea7e2957d3b343448d008d2feb55ab6d/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:13.104041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10023f-b36f-4a69-b4dc-4082c3a2f3a5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:13.208008 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020314df-bf8f-4cff-9d8b-0b192eaed55f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:13.225710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:13.256704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10023f-b36f-4a69-b4dc-4082c3a2f3a5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:13.280560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:13.310810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355d519-1fd2-4968-bd58-7fa6e0091b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:13.310810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355d519-1fd2-4968-bd58-7fa6e0091b22 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:13.310810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355d519-1fd2-4968-bd58-7fa6e0091b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:13.310810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355d519-1fd2-4968-bd58-7fa6e0091b22 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:13.320790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b146e-a0bc-42b0-9d48-697c346f9a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:13.321832 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b146e-a0bc-42b0-9d48-697c346f9a31 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:13.322347 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b146e-a0bc-42b0-9d48-697c346f9a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:13.322881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b146e-a0bc-42b0-9d48-697c346f9a31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:13.343831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355d519-1fd2-4968-bd58-7fa6e0091b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:13.345057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b146e-a0bc-42b0-9d48-697c346f9a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7355d519-1fd2-4968-bd58-7fa6e0091b22 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:13.348765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:13.349734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:13.350615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:13.351939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 125/258] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:46:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-bb2b146e-a0bc-42b0-9d48-697c346f9a31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:13.353660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:13.354531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:13.355678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 134/259] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:46:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:13.360358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355d519-1fd2-4968-bd58-7fa6e0091b22 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:13.360472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b146e-a0bc-42b0-9d48-697c346f9a31 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:13.466843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355d519-1fd2-4968-bd58-7fa6e0091b22 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:13.539811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b146e-a0bc-42b0-9d48-697c346f9a31 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:13.543502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1f07-9ef7-4aa5-83cb-edfca93ed4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:13.544148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1f07-9ef7-4aa5-83cb-edfca93ed4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:13.544431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1f07-9ef7-4aa5-83cb-edfca93ed4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:13.544705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1f07-9ef7-4aa5-83cb-edfca93ed4a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:13.545446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c1f07-9ef7-4aa5-83cb-edfca93ed4a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:13.545753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c1f07-9ef7-4aa5-83cb-edfca93ed4a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:13.547180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c1f07-9ef7-4aa5-83cb-edfca93ed4a3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:13.566351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 126/260] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:13.573711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1f07-9ef7-4aa5-83cb-edfca93ed4a3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:13.627009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ddf9b-5320-4279-b1d0-7ba3d94d9f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:13.629200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ddf9b-5320-4279-b1d0-7ba3d94d9f6e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:13.629721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ddf9b-5320-4279-b1d0-7ba3d94d9f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:13.630201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ddf9b-5320-4279-b1d0-7ba3d94d9f6e None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:13.631743 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ddf9b-5320-4279-b1d0-7ba3d94d9f6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:13.632274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ddf9b-5320-4279-b1d0-7ba3d94d9f6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:13.634324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ddf9b-5320-4279-b1d0-7ba3d94d9f6e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:13.649499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 135/261] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:13.652520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:13.659669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ddf9b-5320-4279-b1d0-7ba3d94d9f6e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:13.706377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:13.815771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ddf9b-5320-4279-b1d0-7ba3d94d9f6e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:13.828836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1f07-9ef7-4aa5-83cb-edfca93ed4a3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:13.858559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191bf2d-2de2-4387-830b-5311c9528271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:13.859257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191bf2d-2de2-4387-830b-5311c9528271 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:13.859653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191bf2d-2de2-4387-830b-5311c9528271 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:13.859911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191bf2d-2de2-4387-830b-5311c9528271 None admin] PATH_INFO: `/v3/users/13526544506140c5a5f223e306a2abd6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:13.870810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191bf2d-2de2-4387-830b-5311c9528271 None admin] RBAC: Authorizing `identity:get_user(user_id=13526544506140c5a5f223e306a2abd6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:13.870810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191bf2d-2de2-4387-830b-5311c9528271 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:13.871149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 127/262] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:13 2022] GET /identity/v3/users/13526544506140c5a5f223e306a2abd6 => generated 283 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:13.875833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f011e13-fe7e-4ee8-80d1-00b3cfb141d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:13.876273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191bf2d-2de2-4387-830b-5311c9528271 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:13.877009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f011e13-fe7e-4ee8-80d1-00b3cfb141d8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:13.877483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f011e13-fe7e-4ee8-80d1-00b3cfb141d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:13.877944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f011e13-fe7e-4ee8-80d1-00b3cfb141d8 None admin] PATH_INFO: `/v3/users/13526544506140c5a5f223e306a2abd6` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:13.880116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f011e13-fe7e-4ee8-80d1-00b3cfb141d8 None admin] RBAC: Authorizing `identity:get_user(user_id=13526544506140c5a5f223e306a2abd6)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:13.881850 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f011e13-fe7e-4ee8-80d1-00b3cfb141d8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:13.885507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 136/263] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:13 2022] GET /identity/v3/users/13526544506140c5a5f223e306a2abd6 => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:13.912020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f011e13-fe7e-4ee8-80d1-00b3cfb141d8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:14.019326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:14.028012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191bf2d-2de2-4387-830b-5311c9528271 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:14.092779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a92fc-a082-4314-b4f3-5850944bc0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:14.093327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a92fc-a082-4314-b4f3-5850944bc0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:14.093613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a92fc-a082-4314-b4f3-5850944bc0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:14.093890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a92fc-a082-4314-b4f3-5850944bc0f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:14.195763 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f011e13-fe7e-4ee8-80d1-00b3cfb141d8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:14.222626 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:14.247464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a92fc-a082-4314-b4f3-5850944bc0f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:14.249630 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a92fc-a082-4314-b4f3-5850944bc0f7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:14.299499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0c3b5-af60-4960-bfb5-d0aacb2d6477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:14.300142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0c3b5-af60-4960-bfb5-d0aacb2d6477 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:14.300427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0c3b5-af60-4960-bfb5-d0aacb2d6477 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:14.300706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0c3b5-af60-4960-bfb5-d0aacb2d6477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:14.447905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:14.460410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 128/264] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:14.465649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a92fc-a082-4314-b4f3-5850944bc0f7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:14.473472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0c3b5-af60-4960-bfb5-d0aacb2d6477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:14.475526 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0c3b5-af60-4960-bfb5-d0aacb2d6477 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:14.679508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 137/265] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 768 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 03:46:14.680749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a92fc-a082-4314-b4f3-5850944bc0f7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:14.685700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0c3b5-af60-4960-bfb5-d0aacb2d6477 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:14.692039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:14.752998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79de25-0f55-4bf6-b606-2217861b9440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:14.753583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79de25-0f55-4bf6-b606-2217861b9440 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:14.753870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79de25-0f55-4bf6-b606-2217861b9440 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:14.754149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79de25-0f55-4bf6-b606-2217861b9440 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:14.759579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79de25-0f55-4bf6-b606-2217861b9440 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:14.761053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79de25-0f55-4bf6-b606-2217861b9440 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:14.764472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 129/266] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:14 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:14.770806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79de25-0f55-4bf6-b606-2217861b9440 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:14.857629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0c3b5-af60-4960-bfb5-d0aacb2d6477 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:14.882597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:14.894266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7803d4a-b586-4c45-9fb4-1b963a144ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:14.895358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7803d4a-b586-4c45-9fb4-1b963a144ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:14.895846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7803d4a-b586-4c45-9fb4-1b963a144ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:14.896317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7803d4a-b586-4c45-9fb4-1b963a144ea8 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:14.898343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7803d4a-b586-4c45-9fb4-1b963a144ea8 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:14.900091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7803d4a-b586-4c45-9fb4-1b963a144ea8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:14.902479 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 138/267] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:14 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:14.904212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22180-a8ad-4d8b-95d7-13d6ecd08729 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:14.904704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22180-a8ad-4d8b-95d7-13d6ecd08729 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:14.905189 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22180-a8ad-4d8b-95d7-13d6ecd08729 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:14.910565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 139/268] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:14.913164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf5550-d895-4d11-9243-06aec7101fd2 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:14.913729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf5550-d895-4d11-9243-06aec7101fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:14.914196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf5550-d895-4d11-9243-06aec7101fd2 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:14.915994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 140/269] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:14.916947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf5550-d895-4d11-9243-06aec7101fd2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:14.946215 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79de25-0f55-4bf6-b606-2217861b9440 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:14.983046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057bbeb-1570-413b-bb43-2505f0ead5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:14.983678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057bbeb-1570-413b-bb43-2505f0ead5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:14.984003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057bbeb-1570-413b-bb43-2505f0ead5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:14.984237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057bbeb-1570-413b-bb43-2505f0ead5c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:14.985040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4057bbeb-1570-413b-bb43-2505f0ead5c5 None admin] RBAC: Adding query filter params (role.id=854a7f7c663b4c66a08f7e731c7bf63c, scope.project.id=6409fa7c592e4bd09561cddf768267d4, user.id=13526544506140c5a5f223e306a2abd6) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:14.985352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4057bbeb-1570-413b-bb43-2505f0ead5c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:14.986817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4057bbeb-1570-413b-bb43-2505f0ead5c5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:14.993419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 130/270] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:14 2022] GET /identity/v3/role_assignments?user.id=13526544506140c5a5f223e306a2abd6&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=854a7f7c663b4c66a08f7e731c7bf63c => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:14.994642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f432d-cde6-4f9c-a254-7a214f1526b7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:14.994954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f432d-cde6-4f9c-a254-7a214f1526b7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:14.995240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f432d-cde6-4f9c-a254-7a214f1526b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.011599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:15.027376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ee9f432d-cde6-4f9c-a254-7a214f1526b7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:15.028645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ee9f432d-cde6-4f9c-a254-7a214f1526b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:15.048426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf5550-d895-4d11-9243-06aec7101fd2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:15.089245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5326d9a-ce1a-4ae2-9e6b-66e07e22f964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:15.090416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5326d9a-ce1a-4ae2-9e6b-66e07e22f964 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.090926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5326d9a-ce1a-4ae2-9e6b-66e07e22f964 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.091489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5326d9a-ce1a-4ae2-9e6b-66e07e22f964 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.092625 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5326d9a-ce1a-4ae2-9e6b-66e07e22f964 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=6409fa7c592e4bd09561cddf768267d4, user.id=13526544506140c5a5f223e306a2abd6) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:15.093225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5326d9a-ce1a-4ae2-9e6b-66e07e22f964 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:15.095043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5326d9a-ce1a-4ae2-9e6b-66e07e22f964 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:15.103289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 141/271] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:14 2022] GET /identity/v3/role_assignments?user.id=13526544506140c5a5f223e306a2abd6&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=a37ae24c516b48c69e13721c013134a2 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:15.105176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7782ef-7620-40ba-989f-f894083b2d59 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.105714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7782ef-7620-40ba-989f-f894083b2d59 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.106179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7782ef-7620-40ba-989f-f894083b2d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.127144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 131/272] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:15.136737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408a44-9790-4d1d-9874-5eba03791723 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.137280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408a44-9790-4d1d-9874-5eba03791723 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.137819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408a44-9790-4d1d-9874-5eba03791723 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.145615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 132/273] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:15.151803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252c94d-f013-43c7-b321-cd93ce0651e4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.152414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252c94d-f013-43c7-b321-cd93ce0651e4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.152882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252c94d-f013-43c7-b321-cd93ce0651e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.158510 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8d7782ef-7620-40ba-989f-f894083b2d59 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:15.165201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8d7782ef-7620-40ba-989f-f894083b2d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:15.186488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6252c94d-f013-43c7-b321-cd93ce0651e4 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:15.188932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6252c94d-f013-43c7-b321-cd93ce0651e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:15.258632 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:15.263975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 142/274] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:15.264654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7782ef-7620-40ba-989f-f894083b2d59 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:15.276626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 133/275] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:15.291926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252c94d-f013-43c7-b321-cd93ce0651e4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:15.341357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:15.389431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7782ef-7620-40ba-989f-f894083b2d59 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:15.430495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252c94d-f013-43c7-b321-cd93ce0651e4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:15.432870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a2ecf-5155-4ab1-8e8f-11daa31c74b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:15.433522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a2ecf-5155-4ab1-8e8f-11daa31c74b5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.433807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a2ecf-5155-4ab1-8e8f-11daa31c74b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.434079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a2ecf-5155-4ab1-8e8f-11daa31c74b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.450267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a2ecf-5155-4ab1-8e8f-11daa31c74b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-ed0a2ecf-5155-4ab1-8e8f-11daa31c74b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:15.456408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:15.457301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:15.458086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 143/276] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:15.458732 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a2ecf-5155-4ab1-8e8f-11daa31c74b5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:15.471175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa414c8-07e1-4203-b4d5-dc47c9d58cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:15.471828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa414c8-07e1-4203-b4d5-dc47c9d58cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.472108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa414c8-07e1-4203-b4d5-dc47c9d58cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.472381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa414c8-07e1-4203-b4d5-dc47c9d58cb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.473546 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa414c8-07e1-4203-b4d5-dc47c9d58cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:15.475074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa414c8-07e1-4203-b4d5-dc47c9d58cb7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:15.478852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 134/277] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:15.478852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa414c8-07e1-4203-b4d5-dc47c9d58cb7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:15.545779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:15.578167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a2ecf-5155-4ab1-8e8f-11daa31c74b5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:15.591430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa414c8-07e1-4203-b4d5-dc47c9d58cb7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:15.605959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:15.616523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2725e-1c7b-46f6-9e9d-9dffda7cb5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:15.617362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2725e-1c7b-46f6-9e9d-9dffda7cb5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.617779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2725e-1c7b-46f6-9e9d-9dffda7cb5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.618229 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2725e-1c7b-46f6-9e9d-9dffda7cb5d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.628740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2725e-1c7b-46f6-9e9d-9dffda7cb5d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:15.628866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84d501-fd8e-4d6d-913f-9ec4cf616eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:15.629366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84d501-fd8e-4d6d-913f-9ec4cf616eaa None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.629663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84d501-fd8e-4d6d-913f-9ec4cf616eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.629942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84d501-fd8e-4d6d-913f-9ec4cf616eaa None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.630664 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84d501-fd8e-4d6d-913f-9ec4cf616eaa None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:15.631089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84d501-fd8e-4d6d-913f-9ec4cf616eaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:15.632475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84d501-fd8e-4d6d-913f-9ec4cf616eaa None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-24c2725e-1c7b-46f6-9e9d-9dffda7cb5d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:15.633857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:15.634692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:15.635531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 144/278] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:15.636184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2725e-1c7b-46f6-9e9d-9dffda7cb5d4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:15.643913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 135/279] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:15.644720 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84d501-fd8e-4d6d-913f-9ec4cf616eaa None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:15.752670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2725e-1c7b-46f6-9e9d-9dffda7cb5d4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:15.756627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84d501-fd8e-4d6d-913f-9ec4cf616eaa None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:15.775290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:15.790695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db5790-2789-41a6-b6ef-256d7949de99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:15.791277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db5790-2789-41a6-b6ef-256d7949de99 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.791564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db5790-2789-41a6-b6ef-256d7949de99 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.791879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db5790-2789-41a6-b6ef-256d7949de99 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.792518 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db5790-2789-41a6-b6ef-256d7949de99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:15.793976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db5790-2789-41a6-b6ef-256d7949de99 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:15.794269 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cf9b2-4a04-45c3-bb9a-6acdb0eb9649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:15.794922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cf9b2-4a04-45c3-bb9a-6acdb0eb9649 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.795210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cf9b2-4a04-45c3-bb9a-6acdb0eb9649 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.795491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cf9b2-4a04-45c3-bb9a-6acdb0eb9649 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.796217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cf9b2-4a04-45c3-bb9a-6acdb0eb9649 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:15.796526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cf9b2-4a04-45c3-bb9a-6acdb0eb9649 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:15.797891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cf9b2-4a04-45c3-bb9a-6acdb0eb9649 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:15.807960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 136/280] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:15.808808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cf9b2-4a04-45c3-bb9a-6acdb0eb9649 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:15.830815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 145/281] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:46:15 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:46:15.830815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db5790-2789-41a6-b6ef-256d7949de99 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:15.892216 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:15.955312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cf9b2-4a04-45c3-bb9a-6acdb0eb9649 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:15.955899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db5790-2789-41a6-b6ef-256d7949de99 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:15.995780 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8a1f4-4ced-4ac6-99d6-6dc2ffdf17e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:15.996870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8a1f4-4ced-4ac6-99d6-6dc2ffdf17e5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.996870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8a1f4-4ced-4ac6-99d6-6dc2ffdf17e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.997053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8a1f4-4ced-4ac6-99d6-6dc2ffdf17e5 None admin] PATH_INFO: `/v3/users/f5eafcfd910949638321d3a415669581` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.997131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a98d9-3df9-438b-b1c4-0fb0f8381a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:15.998591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a98d9-3df9-438b-b1c4-0fb0f8381a5e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:15.998591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a98d9-3df9-438b-b1c4-0fb0f8381a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:15.998748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8a1f4-4ced-4ac6-99d6-6dc2ffdf17e5 None admin] RBAC: Authorizing `identity:get_user(user_id=f5eafcfd910949638321d3a415669581)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:15.998860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a98d9-3df9-438b-b1c4-0fb0f8381a5e None admin] PATH_INFO: `/v3/users/8528e7e46ad1478ebf804b799c72bf50` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:15.999913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8a1f4-4ced-4ac6-99d6-6dc2ffdf17e5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:16.004300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 146/282] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:15 2022] GET /identity/v3/users/f5eafcfd910949638321d3a415669581 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:16.004300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac46a23-e0cc-47ab-bc38-7e94d2cb8be5 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:16.004300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac46a23-e0cc-47ab-bc38-7e94d2cb8be5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:16.004300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac46a23-e0cc-47ab-bc38-7e94d2cb8be5 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:16.004300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 147/283] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:16.008113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac46a23-e0cc-47ab-bc38-7e94d2cb8be5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:16.015694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a98d9-3df9-438b-b1c4-0fb0f8381a5e None admin] RBAC: Authorizing `identity:get_user(user_id=8528e7e46ad1478ebf804b799c72bf50)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:16.017185 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a98d9-3df9-438b-b1c4-0fb0f8381a5e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:16.018939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 137/284] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:15 2022] GET /identity/v3/users/8528e7e46ad1478ebf804b799c72bf50 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:16.025457 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebe240-f3c5-4afd-b7e6-b894ba1ef7ea None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:16.025747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebe240-f3c5-4afd-b7e6-b894ba1ef7ea None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:16.026028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebe240-f3c5-4afd-b7e6-b894ba1ef7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:16.045010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-35ebe240-f3c5-4afd-b7e6-b894ba1ef7ea None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:16.046206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-35ebe240-f3c5-4afd-b7e6-b894ba1ef7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:16.084417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:16.127412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac46a23-e0cc-47ab-bc38-7e94d2cb8be5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:16.134446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 138/285] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:16.135535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebe240-f3c5-4afd-b7e6-b894ba1ef7ea None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:16.139725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:16.171330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff2866-19e3-4eb6-8e08-00f5d33c5a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:16.171981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff2866-19e3-4eb6-8e08-00f5d33c5a9f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:16.172269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff2866-19e3-4eb6-8e08-00f5d33c5a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:16.172551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff2866-19e3-4eb6-8e08-00f5d33c5a9f None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:16.173713 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff2866-19e3-4eb6-8e08-00f5d33c5a9f None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:16.175816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff2866-19e3-4eb6-8e08-00f5d33c5a9f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:16.178381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 148/286] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:16 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:16.178568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff2866-19e3-4eb6-8e08-00f5d33c5a9f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:16.257643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebe240-f3c5-4afd-b7e6-b894ba1ef7ea None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:16.296385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c075b-988c-48f5-aafd-b9d23f398a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:16.296924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c075b-988c-48f5-aafd-b9d23f398a08 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:16.297255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c075b-988c-48f5-aafd-b9d23f398a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:16.297587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c075b-988c-48f5-aafd-b9d23f398a08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:16.298729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c075b-988c-48f5-aafd-b9d23f398a08 None admin] RBAC: Adding query filter params (role.id=f66cbbcc2491438184f04522c2a68578, user.id=8528e7e46ad1478ebf804b799c72bf50, scope.system=all) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:16.298729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c075b-988c-48f5-aafd-b9d23f398a08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:16.300165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c075b-988c-48f5-aafd-b9d23f398a08 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:16.307036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 139/287] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Jul 28 03:46:16 2022] GET /identity/v3/role_assignments?user.id=8528e7e46ad1478ebf804b799c72bf50&scope.system=all&role.id=f66cbbcc2491438184f04522c2a68578 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:16.307859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c075b-988c-48f5-aafd-b9d23f398a08 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:16.312974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff2866-19e3-4eb6-8e08-00f5d33c5a9f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:16.345291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:16.358685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9e8b8-36fb-4ad7-bcc1-6448529faf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:16.367794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9e8b8-36fb-4ad7-bcc1-6448529faf1f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:16.368080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9e8b8-36fb-4ad7-bcc1-6448529faf1f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:16.368354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9e8b8-36fb-4ad7-bcc1-6448529faf1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:16.423510 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9e8b8-36fb-4ad7-bcc1-6448529faf1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e1c9e8b8-36fb-4ad7-bcc1-6448529faf1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:16.429473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:16.430405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:16.431218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:16.432288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c075b-988c-48f5-aafd-b9d23f398a08 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:16.433516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 149/288] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:16.438538 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9e8b8-36fb-4ad7-bcc1-6448529faf1f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:16.477526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f427d9-11ca-4998-bbe7-48737cb283dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:16.478256 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f427d9-11ca-4998-bbe7-48737cb283dc None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:16.478545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f427d9-11ca-4998-bbe7-48737cb283dc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:16.478921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f427d9-11ca-4998-bbe7-48737cb283dc None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4/users/f5eafcfd910949638321d3a415669581/roles/77b5c5dec7334e27a8126a96b7b17f32` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:16.481009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f427d9-11ca-4998-bbe7-48737cb283dc None admin] RBAC: Authorizing `identity:create_grant(project_id=6409fa7c592e4bd09561cddf768267d4, user_id=f5eafcfd910949638321d3a415669581, role_id=77b5c5dec7334e27a8126a96b7b17f32)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:16.482605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f427d9-11ca-4998-bbe7-48737cb283dc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:16.503345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 140/289] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:46:16 2022] PUT /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4/users/f5eafcfd910949638321d3a415669581/roles/77b5c5dec7334e27a8126a96b7b17f32 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:16.521894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:16.605431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9e8b8-36fb-4ad7-bcc1-6448529faf1f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:16.650269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdac2b0-daac-4a08-bc56-18bb92e8821c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:16.651124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdac2b0-daac-4a08-bc56-18bb92e8821c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:16.651415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdac2b0-daac-4a08-bc56-18bb92e8821c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:16.651691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdac2b0-daac-4a08-bc56-18bb92e8821c None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:16.652480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdac2b0-daac-4a08-bc56-18bb92e8821c None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:16.652833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdac2b0-daac-4a08-bc56-18bb92e8821c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:16.654244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdac2b0-daac-4a08-bc56-18bb92e8821c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:16.665244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 150/290] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:16.678480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f427d9-11ca-4998-bbe7-48737cb283dc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:16.822620 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:16.841243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f427d9-11ca-4998-bbe7-48737cb283dc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:16.901322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b66958-5b79-4c61-9e7a-ed9ce23502d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:16.902289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b66958-5b79-4c61-9e7a-ed9ce23502d3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:16.902772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b66958-5b79-4c61-9e7a-ed9ce23502d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:16.903264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b66958-5b79-4c61-9e7a-ed9ce23502d3 None admin] PATH_INFO: `/v3/users/ea7e2957d3b343448d008d2feb55ab6d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:16.907486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b66958-5b79-4c61-9e7a-ed9ce23502d3 None admin] RBAC: Authorizing `identity:get_user(user_id=ea7e2957d3b343448d008d2feb55ab6d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:16.909182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b66958-5b79-4c61-9e7a-ed9ce23502d3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:16.911036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 141/291] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:16 2022] GET /identity/v3/users/ea7e2957d3b343448d008d2feb55ab6d => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:16.917081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdac2b0-daac-4a08-bc56-18bb92e8821c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:17.055448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:17.059101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdac2b0-daac-4a08-bc56-18bb92e8821c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:17.104064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f729db5-278c-4d09-a792-e6a5b37762b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:17.105102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f729db5-278c-4d09-a792-e6a5b37762b5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:17.105590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f729db5-278c-4d09-a792-e6a5b37762b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:17.106070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f729db5-278c-4d09-a792-e6a5b37762b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:17.242624 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f729db5-278c-4d09-a792-e6a5b37762b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:17.245436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f729db5-278c-4d09-a792-e6a5b37762b5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:17.407855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:17.415847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 151/292] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:17.423635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b66958-5b79-4c61-9e7a-ed9ce23502d3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:17.583538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b66958-5b79-4c61-9e7a-ed9ce23502d3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:17.640884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46e0d6-e77f-49ab-8d3e-98ccd7a82031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:17.641534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46e0d6-e77f-49ab-8d3e-98ccd7a82031 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:17.641815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46e0d6-e77f-49ab-8d3e-98ccd7a82031 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:17.642088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46e0d6-e77f-49ab-8d3e-98ccd7a82031 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:17.643317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46e0d6-e77f-49ab-8d3e-98ccd7a82031 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:17.644758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46e0d6-e77f-49ab-8d3e-98ccd7a82031 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:17.647461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 142/293] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:17 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:17.656013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f729db5-278c-4d09-a792-e6a5b37762b5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:17.828388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f729db5-278c-4d09-a792-e6a5b37762b5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:17.880055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb849a04-19df-4f04-816d-c674a0cf5df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:17.880679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849a04-19df-4f04-816d-c674a0cf5df1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:17.881750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849a04-19df-4f04-816d-c674a0cf5df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:17.882112 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849a04-19df-4f04-816d-c674a0cf5df1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:17.882932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb849a04-19df-4f04-816d-c674a0cf5df1 None admin] RBAC: Adding query filter params (role.id=f66cbbcc2491438184f04522c2a68578, scope.project.id=6409fa7c592e4bd09561cddf768267d4, user.id=ea7e2957d3b343448d008d2feb55ab6d) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:17.883242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb849a04-19df-4f04-816d-c674a0cf5df1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:17.884677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb849a04-19df-4f04-816d-c674a0cf5df1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:17.895842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 152/294] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:17 2022] GET /identity/v3/role_assignments?user.id=ea7e2957d3b343448d008d2feb55ab6d&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=f66cbbcc2491438184f04522c2a68578 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:18.614201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cf905-3ac6-4bf4-ad7e-8707ecd3bb58 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:18.615022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cf905-3ac6-4bf4-ad7e-8707ecd3bb58 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:18.615022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cf905-3ac6-4bf4-ad7e-8707ecd3bb58 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:18.616924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 143/295] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:18.624373 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c3640-ab78-43a3-bb8c-769b3a0ed05a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:18.625226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c3640-ab78-43a3-bb8c-769b3a0ed05a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:18.625720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c3640-ab78-43a3-bb8c-769b3a0ed05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:18.647381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ae5c3640-ab78-43a3-bb8c-769b3a0ed05a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:18.649082 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ae5c3640-ab78-43a3-bb8c-769b3a0ed05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:18.733616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:18.744663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 153/296] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:18.766683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cf905-3ac6-4bf4-ad7e-8707ecd3bb58 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:18.780436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f4bce-0e5a-4f28-ae1a-c08f5fb4f85f None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:18.780959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f4bce-0e5a-4f28-ae1a-c08f5fb4f85f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:18.781370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f4bce-0e5a-4f28-ae1a-c08f5fb4f85f None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:18.783259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 154/297] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:18.789631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e9442-476d-42c7-80a5-7651ab3d3d47 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:18.790069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e9442-476d-42c7-80a5-7651ab3d3d47 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:18.790498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e9442-476d-42c7-80a5-7651ab3d3d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:18.798078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:18.812815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cb0e9442-476d-42c7-80a5-7651ab3d3d47 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:18.814203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cb0e9442-476d-42c7-80a5-7651ab3d3d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:18.891083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 155/298] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:18.897066 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cf905-3ac6-4bf4-ad7e-8707ecd3bb58 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:18.915693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e9442-476d-42c7-80a5-7651ab3d3d47 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:18.934555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359ef4f-4f80-4a47-a621-84bc92e64a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:18.935234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359ef4f-4f80-4a47-a621-84bc92e64a83 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:18.935518 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359ef4f-4f80-4a47-a621-84bc92e64a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:18.935803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359ef4f-4f80-4a47-a621-84bc92e64a83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:18.950875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359ef4f-4f80-4a47-a621-84bc92e64a83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b359ef4f-4f80-4a47-a621-84bc92e64a83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:18.957234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:18.958142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:18.959548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 144/299] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:18.964744 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359ef4f-4f80-4a47-a621-84bc92e64a83 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:19.042822 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e9442-476d-42c7-80a5-7651ab3d3d47 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:19.061515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:19.072378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:19.076835 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359ef4f-4f80-4a47-a621-84bc92e64a83 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:19.083360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def4b30-c933-4299-9595-c1687231224a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:19.084529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def4b30-c933-4299-9595-c1687231224a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.084812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def4b30-c933-4299-9595-c1687231224a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.085325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def4b30-c933-4299-9595-c1687231224a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.086223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def4b30-c933-4299-9595-c1687231224a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:19.087722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def4b30-c933-4299-9595-c1687231224a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:19.094024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 156/300] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:19.095347 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866607-d874-47cb-bd0d-c739114a6fa3 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.095497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866607-d874-47cb-bd0d-c739114a6fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.095802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866607-d874-47cb-bd0d-c739114a6fa3 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.098881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 157/301] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:19.098881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60866607-d874-47cb-bd0d-c739114a6fa3 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:19.118362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0a575-aeb1-4f46-a9d0-44f8185af936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:19.119022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0a575-aeb1-4f46-a9d0-44f8185af936 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.119312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0a575-aeb1-4f46-a9d0-44f8185af936 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.119588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0a575-aeb1-4f46-a9d0-44f8185af936 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.120316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0a575-aeb1-4f46-a9d0-44f8185af936 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:19.120622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0a575-aeb1-4f46-a9d0-44f8185af936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:19.121999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0a575-aeb1-4f46-a9d0-44f8185af936 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:19.134872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 145/302] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:19.134872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6800e-6fb3-48c4-bbd4-15828797df6e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.134872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6800e-6fb3-48c4-bbd4-15828797df6e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.134872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6800e-6fb3-48c4-bbd4-15828797df6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.151135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ada6800e-6fb3-48c4-bbd4-15828797df6e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:19.152373 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ada6800e-6fb3-48c4-bbd4-15828797df6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:19.218688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60866607-d874-47cb-bd0d-c739114a6fa3 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:19.225244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 146/303] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:19.226086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6800e-6fb3-48c4-bbd4-15828797df6e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:19.269832 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3e1d2-5287-463a-9711-7b392d04320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:19.270487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3e1d2-5287-463a-9711-7b392d04320e None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.270768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3e1d2-5287-463a-9711-7b392d04320e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.271812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3e1d2-5287-463a-9711-7b392d04320e None admin] PATH_INFO: `/v3/groups` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.272277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3e1d2-5287-463a-9711-7b392d04320e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:19.273703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3e1d2-5287-463a-9711-7b392d04320e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:19.302948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 158/304] 10.222.0.46 () {60 vars in 1134 bytes} [Thu Jul 28 03:46:19 2022] POST /identity/v3/groups => generated 229 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:46:19.302948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3e1d2-5287-463a-9711-7b392d04320e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:19.326073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:19.348489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6800e-6fb3-48c4-bbd4-15828797df6e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:19.369273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:19.390055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1d3f3-800a-4579-ad3a-62673e6c9550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:19.390711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1d3f3-800a-4579-ad3a-62673e6c9550 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.391032 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1d3f3-800a-4579-ad3a-62673e6c9550 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.391310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1d3f3-800a-4579-ad3a-62673e6c9550 None admin] PATH_INFO: `/v3/users/3f5d8dd7849445c6bd56700e0ad156c3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.407709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1d3f3-800a-4579-ad3a-62673e6c9550 None admin] RBAC: Authorizing `identity:get_user(user_id=3f5d8dd7849445c6bd56700e0ad156c3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:19.409221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1d3f3-800a-4579-ad3a-62673e6c9550 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:19.416280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 147/305] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:19 2022] GET /identity/v3/users/3f5d8dd7849445c6bd56700e0ad156c3 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:19.416280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ee86b-c7f5-4902-9027-ae230aaf2811 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.416280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ee86b-c7f5-4902-9027-ae230aaf2811 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.416280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ee86b-c7f5-4902-9027-ae230aaf2811 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.419118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 148/306] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:19.419948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ee86b-c7f5-4902-9027-ae230aaf2811 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:19.434357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3e1d2-5287-463a-9711-7b392d04320e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:19.474179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de955566-2e6e-4642-9f92-eab167101a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:19.475498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de955566-2e6e-4642-9f92-eab167101a9c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.475777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de955566-2e6e-4642-9f92-eab167101a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.476048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de955566-2e6e-4642-9f92-eab167101a9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.488327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de955566-2e6e-4642-9f92-eab167101a9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-de955566-2e6e-4642-9f92-eab167101a9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:19.494467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:19.495368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:19.496397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 159/307] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:19.497049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240d2eb-661a-4136-a410-75bf2afd6f54 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.497113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240d2eb-661a-4136-a410-75bf2afd6f54 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.497887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240d2eb-661a-4136-a410-75bf2afd6f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.515866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6240d2eb-661a-4136-a410-75bf2afd6f54 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:19.517053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6240d2eb-661a-4136-a410-75bf2afd6f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:19.540886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ee86b-c7f5-4902-9027-ae230aaf2811 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:19.582479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8c582-7a39-4286-b183-2bbd4e7dc27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:19.583462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8c582-7a39-4286-b183-2bbd4e7dc27e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.583767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8c582-7a39-4286-b183-2bbd4e7dc27e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.584176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8c582-7a39-4286-b183-2bbd4e7dc27e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.584969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8c582-7a39-4286-b183-2bbd4e7dc27e None admin] RBAC: Adding query filter params (role.id=77b5c5dec7334e27a8126a96b7b17f32, user.id=3f5d8dd7849445c6bd56700e0ad156c3, scope.system=all) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:19.585281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8c582-7a39-4286-b183-2bbd4e7dc27e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:19.586722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8c582-7a39-4286-b183-2bbd4e7dc27e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:19.592057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 149/308] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Jul 28 03:46:19 2022] GET /identity/v3/role_assignments?user.id=3f5d8dd7849445c6bd56700e0ad156c3&scope.system=all&role.id=77b5c5dec7334e27a8126a96b7b17f32 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:19.592840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8c582-7a39-4286-b183-2bbd4e7dc27e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:19.595103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 160/309] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:19.600383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:19.621516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240d2eb-661a-4136-a410-75bf2afd6f54 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:19.691141 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:19.727772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8c582-7a39-4286-b183-2bbd4e7dc27e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:19.775574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240d2eb-661a-4136-a410-75bf2afd6f54 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:19.778761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71b38d-1ba4-40f9-89b6-7019973cb9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:19.779682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71b38d-1ba4-40f9-89b6-7019973cb9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.780162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71b38d-1ba4-40f9-89b6-7019973cb9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.780646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71b38d-1ba4-40f9-89b6-7019973cb9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.781564 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71b38d-1ba4-40f9-89b6-7019973cb9c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:19.782064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71b38d-1ba4-40f9-89b6-7019973cb9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:19.784213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71b38d-1ba4-40f9-89b6-7019973cb9c9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:19.797253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 150/310] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:19.804157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71b38d-1ba4-40f9-89b6-7019973cb9c9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:19.818081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737de81-6022-4c81-b53b-96d15b3afb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:19.819767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737de81-6022-4c81-b53b-96d15b3afb54 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.820270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737de81-6022-4c81-b53b-96d15b3afb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.820871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737de81-6022-4c81-b53b-96d15b3afb54 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.832933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737de81-6022-4c81-b53b-96d15b3afb54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-2737de81-6022-4c81-b53b-96d15b3afb54 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:19.839798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:19.840692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:19.841460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:19.848894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 161/311] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:19.850905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737de81-6022-4c81-b53b-96d15b3afb54 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:19.869828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:19.909222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71b38d-1ba4-40f9-89b6-7019973cb9c9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:19.945245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0a1e4-f678-4b3d-8974-574b9bbcbf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:19.945892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0a1e4-f678-4b3d-8974-574b9bbcbf1f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:19.946173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0a1e4-f678-4b3d-8974-574b9bbcbf1f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:19.946447 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0a1e4-f678-4b3d-8974-574b9bbcbf1f None admin] PATH_INFO: `/v3/users/8528e7e46ad1478ebf804b799c72bf50` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:19.947904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0a1e4-f678-4b3d-8974-574b9bbcbf1f None admin] RBAC: Authorizing `identity:get_user(user_id=8528e7e46ad1478ebf804b799c72bf50)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:19.949141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0a1e4-f678-4b3d-8974-574b9bbcbf1f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:19.951083 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 151/312] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:19 2022] GET /identity/v3/users/8528e7e46ad1478ebf804b799c72bf50 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:19.956107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0a1e4-f678-4b3d-8974-574b9bbcbf1f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:19.983126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737de81-6022-4c81-b53b-96d15b3afb54 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:19.985022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:20.024405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d45b9e-35b2-4e69-adfa-2026953e1944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:20.025473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d45b9e-35b2-4e69-adfa-2026953e1944 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:20.026035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d45b9e-35b2-4e69-adfa-2026953e1944 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:20.026510 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d45b9e-35b2-4e69-adfa-2026953e1944 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:20.027533 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d45b9e-35b2-4e69-adfa-2026953e1944 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:20.028161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d45b9e-35b2-4e69-adfa-2026953e1944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:20.029855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d45b9e-35b2-4e69-adfa-2026953e1944 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:20.039877 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 162/313] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:20.045562 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d45b9e-35b2-4e69-adfa-2026953e1944 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:20.081521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0a1e4-f678-4b3d-8974-574b9bbcbf1f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:20.114198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:20.121277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd320a-345a-474f-a184-4ce9a7e06256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:20.122223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd320a-345a-474f-a184-4ce9a7e06256 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:20.122983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd320a-345a-474f-a184-4ce9a7e06256 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:20.123683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd320a-345a-474f-a184-4ce9a7e06256 None admin] PATH_INFO: `/v3/system/users/8528e7e46ad1478ebf804b799c72bf50/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:20.125602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd320a-345a-474f-a184-4ce9a7e06256 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8528e7e46ad1478ebf804b799c72bf50, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:20.127237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd320a-345a-474f-a184-4ce9a7e06256 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:20.140406 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 152/314] 10.222.0.46 () {58 vars in 1334 bytes} [Thu Jul 28 03:46:19 2022] PUT /identity/v3/system/users/8528e7e46ad1478ebf804b799c72bf50/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:20.179013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d45b9e-35b2-4e69-adfa-2026953e1944 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:20.218367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f8bbd-5d0c-49eb-9a35-ee21212be63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:20.219106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f8bbd-5d0c-49eb-9a35-ee21212be63e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:20.219594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f8bbd-5d0c-49eb-9a35-ee21212be63e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:20.219876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f8bbd-5d0c-49eb-9a35-ee21212be63e None admin] PATH_INFO: `/v3/users/f5eafcfd910949638321d3a415669581` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:20.221240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f8bbd-5d0c-49eb-9a35-ee21212be63e None admin] RBAC: Authorizing `identity:get_user(user_id=f5eafcfd910949638321d3a415669581)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:20.223041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f8bbd-5d0c-49eb-9a35-ee21212be63e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:20.226180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 163/315] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:20 2022] GET /identity/v3/users/f5eafcfd910949638321d3a415669581 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:20.232184 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd320a-345a-474f-a184-4ce9a7e06256 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:20.345372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd320a-345a-474f-a184-4ce9a7e06256 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:20.382680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54e0f5-838d-495c-b816-940333e0c393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:20.383257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54e0f5-838d-495c-b816-940333e0c393 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:20.383545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54e0f5-838d-495c-b816-940333e0c393 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:20.383824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54e0f5-838d-495c-b816-940333e0c393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:20.450461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:20.457436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54e0f5-838d-495c-b816-940333e0c393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:20.459088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54e0f5-838d-495c-b816-940333e0c393 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:20.568183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 153/316] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:20.576092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f8bbd-5d0c-49eb-9a35-ee21212be63e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:20.611169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:20.684658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f8bbd-5d0c-49eb-9a35-ee21212be63e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:20.727297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79330481-dd92-44b5-ab07-5c256b964631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:20.727967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79330481-dd92-44b5-ab07-5c256b964631 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:20.728221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79330481-dd92-44b5-ab07-5c256b964631 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:20.728498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79330481-dd92-44b5-ab07-5c256b964631 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:20.729696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79330481-dd92-44b5-ab07-5c256b964631 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:20.731180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79330481-dd92-44b5-ab07-5c256b964631 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:20.733160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 164/317] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:20 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:20.738993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54e0f5-838d-495c-b816-940333e0c393 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:20.841013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:20.846729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54e0f5-838d-495c-b816-940333e0c393 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:20.882903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a692a-4ea7-40cc-bc47-b314033110a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:20.883519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a692a-4ea7-40cc-bc47-b314033110a8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:20.883798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a692a-4ea7-40cc-bc47-b314033110a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:20.884074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a692a-4ea7-40cc-bc47-b314033110a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:20.884859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a692a-4ea7-40cc-bc47-b314033110a8 None admin] RBAC: Adding query filter params (role.id=77b5c5dec7334e27a8126a96b7b17f32, scope.project.id=6409fa7c592e4bd09561cddf768267d4, user.id=f5eafcfd910949638321d3a415669581) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:20.885168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a692a-4ea7-40cc-bc47-b314033110a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:20.886598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a692a-4ea7-40cc-bc47-b314033110a8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:20.892870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 154/318] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:20 2022] GET /identity/v3/role_assignments?user.id=f5eafcfd910949638321d3a415669581&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=77b5c5dec7334e27a8126a96b7b17f32 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:21.360570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99432c7e-b6a6-4d15-850b-7a08c370e8a3 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:21.360934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99432c7e-b6a6-4d15-850b-7a08c370e8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:21.361137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99432c7e-b6a6-4d15-850b-7a08c370e8a3 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:21.362531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 165/319] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:21.367606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7b67d-fda3-4bb1-88c6-960678abae66 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:21.367896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7b67d-fda3-4bb1-88c6-960678abae66 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:21.368168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7b67d-fda3-4bb1-88c6-960678abae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:21.387412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8ea7b67d-fda3-4bb1-88c6-960678abae66 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:21.388643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8ea7b67d-fda3-4bb1-88c6-960678abae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:21.459746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 155/320] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:21.481472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99432c7e-b6a6-4d15-850b-7a08c370e8a3 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:21.583367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:21.588714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99432c7e-b6a6-4d15-850b-7a08c370e8a3 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:21.623294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9c2bc-7893-4520-816c-da5500139c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:21.623864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9c2bc-7893-4520-816c-da5500139c31 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:21.624145 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9c2bc-7893-4520-816c-da5500139c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:21.624421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9c2bc-7893-4520-816c-da5500139c31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:21.625548 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9c2bc-7893-4520-816c-da5500139c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:21.627025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9c2bc-7893-4520-816c-da5500139c31 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:21.628863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 166/321] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:21.634346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7b67d-fda3-4bb1-88c6-960678abae66 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:21.639839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc67079-e7be-4707-a95d-67396fd757e3 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:21.640131 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc67079-e7be-4707-a95d-67396fd757e3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:21.640406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc67079-e7be-4707-a95d-67396fd757e3 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:21.641843 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 167/322] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:46:21.647137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5feaa7-9034-4beb-ae00-05a383cd9ff2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:21.647471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5feaa7-9034-4beb-ae00-05a383cd9ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:21.647746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5feaa7-9034-4beb-ae00-05a383cd9ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:21.665974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-dd5feaa7-9034-4beb-ae00-05a383cd9ff2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:21.667188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dd5feaa7-9034-4beb-ae00-05a383cd9ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:21.732826 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:21.746871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 168/323] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:21.747386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7b67d-fda3-4bb1-88c6-960678abae66 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:21.764098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5feaa7-9034-4beb-ae00-05a383cd9ff2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:21.784010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb8585-d50d-4747-89cc-36b8a4b823db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:21.784657 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb8585-d50d-4747-89cc-36b8a4b823db None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:21.784938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb8585-d50d-4747-89cc-36b8a4b823db None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:21.785210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb8585-d50d-4747-89cc-36b8a4b823db None admin] PATH_INFO: `/v3/groups` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:21.785900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb8585-d50d-4747-89cc-36b8a4b823db None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:21.788363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb8585-d50d-4747-89cc-36b8a4b823db None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:21.807100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 156/324] 10.222.0.46 () {60 vars in 1134 bytes} [Thu Jul 28 03:46:21 2022] POST /identity/v3/groups => generated 226 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:46:21.881394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:21.886108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5feaa7-9034-4beb-ae00-05a383cd9ff2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:21.924446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e782c-8e7d-44cd-84df-85e541e38890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:21.925108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e782c-8e7d-44cd-84df-85e541e38890 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:21.925388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e782c-8e7d-44cd-84df-85e541e38890 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:21.925659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e782c-8e7d-44cd-84df-85e541e38890 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:21.937824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e782c-8e7d-44cd-84df-85e541e38890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-f32e782c-8e7d-44cd-84df-85e541e38890 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:21.942804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:21.944785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:21.945575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:21.947116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 169/325] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:21.950678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb8585-d50d-4747-89cc-36b8a4b823db None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:22.071771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb8585-d50d-4747-89cc-36b8a4b823db None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:22.114532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730767a-ee42-4476-b0c5-b68b1a9a8682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:22.116099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730767a-ee42-4476-b0c5-b68b1a9a8682 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:22.116386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730767a-ee42-4476-b0c5-b68b1a9a8682 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:22.116663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730767a-ee42-4476-b0c5-b68b1a9a8682 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:22.117392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730767a-ee42-4476-b0c5-b68b1a9a8682 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:22.118111 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730767a-ee42-4476-b0c5-b68b1a9a8682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:22.119183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730767a-ee42-4476-b0c5-b68b1a9a8682 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:22.128252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 157/326] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:22.134361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e782c-8e7d-44cd-84df-85e541e38890 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:22.250489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e782c-8e7d-44cd-84df-85e541e38890 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:22.287132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f369447-98d6-4e0c-8499-7a8d15a2d80c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:22.287517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f369447-98d6-4e0c-8499-7a8d15a2d80c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:22.287892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f369447-98d6-4e0c-8499-7a8d15a2d80c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:22.289167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 158/327] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:46:22.289267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe264cf6-732b-4ca2-8356-73ab580ca0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:22.289752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264cf6-732b-4ca2-8356-73ab580ca0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:22.290048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264cf6-732b-4ca2-8356-73ab580ca0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:22.290327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264cf6-732b-4ca2-8356-73ab580ca0c2 None admin] PATH_INFO: `/v3/users/3f5d8dd7849445c6bd56700e0ad156c3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:22.291596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe264cf6-732b-4ca2-8356-73ab580ca0c2 None admin] RBAC: Authorizing `identity:get_user(user_id=3f5d8dd7849445c6bd56700e0ad156c3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:22.293074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe264cf6-732b-4ca2-8356-73ab580ca0c2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:22.294837 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 170/328] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:22 2022] GET /identity/v3/users/3f5d8dd7849445c6bd56700e0ad156c3 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:22.295667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f1e2-73e6-4712-9295-259b7f3a7b97 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:22.295897 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f1e2-73e6-4712-9295-259b7f3a7b97 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:22.296176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f1e2-73e6-4712-9295-259b7f3a7b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:22.300480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe264cf6-732b-4ca2-8356-73ab580ca0c2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:22.325253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3a98f1e2-73e6-4712-9295-259b7f3a7b97 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:22.326756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3a98f1e2-73e6-4712-9295-259b7f3a7b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:22.337512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:22.343570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:22.413613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 159/329] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:22.428985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe264cf6-732b-4ca2-8356-73ab580ca0c2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:22.437997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f1e2-73e6-4712-9295-259b7f3a7b97 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:22.467320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1480a3-12c1-4a6b-9387-cbd183defb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:22.468140 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1480a3-12c1-4a6b-9387-cbd183defb02 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:22.468428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1480a3-12c1-4a6b-9387-cbd183defb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:22.468706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1480a3-12c1-4a6b-9387-cbd183defb02 None admin] PATH_INFO: `/v3/system/users/3f5d8dd7849445c6bd56700e0ad156c3/roles/77b5c5dec7334e27a8126a96b7b17f32` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:22.470334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1480a3-12c1-4a6b-9387-cbd183defb02 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3f5d8dd7849445c6bd56700e0ad156c3, role_id=77b5c5dec7334e27a8126a96b7b17f32)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:22.471873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1480a3-12c1-4a6b-9387-cbd183defb02 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:22.483748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 171/330] 10.222.0.46 () {58 vars in 1334 bytes} [Thu Jul 28 03:46:22 2022] PUT /identity/v3/system/users/3f5d8dd7849445c6bd56700e0ad156c3/roles/77b5c5dec7334e27a8126a96b7b17f32 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 03:46:22.564966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f1e2-73e6-4712-9295-259b7f3a7b97 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:22.605734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5bda6-8f81-4484-97af-89422524498b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:22.606408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bda6-8f81-4484-97af-89422524498b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:22.606693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bda6-8f81-4484-97af-89422524498b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:22.607060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bda6-8f81-4484-97af-89422524498b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:22.620622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b5bda6-8f81-4484-97af-89422524498b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f8b5bda6-8f81-4484-97af-89422524498b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:22.625952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:22.626909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:22.627689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 160/331] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:22.633789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1480a3-12c1-4a6b-9387-cbd183defb02 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:22.747871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:22.753966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1480a3-12c1-4a6b-9387-cbd183defb02 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:22.791873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fe813-c85a-47f7-b933-6fde37e70a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:22.792494 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fe813-c85a-47f7-b933-6fde37e70a40 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:22.792778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fe813-c85a-47f7-b933-6fde37e70a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:22.793053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fe813-c85a-47f7-b933-6fde37e70a40 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:22.793784 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fe813-c85a-47f7-b933-6fde37e70a40 None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:22.794092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fe813-c85a-47f7-b933-6fde37e70a40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:22.795500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fe813-c85a-47f7-b933-6fde37e70a40 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:22.805180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 172/332] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:22.810470 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5bda6-8f81-4484-97af-89422524498b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:22.813968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:22.921497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5bda6-8f81-4484-97af-89422524498b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:22.961511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9013e-c50b-44f4-9dbe-aaf90d993006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:22.962209 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9013e-c50b-44f4-9dbe-aaf90d993006 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:22.962495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9013e-c50b-44f4-9dbe-aaf90d993006 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:22.962770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9013e-c50b-44f4-9dbe-aaf90d993006 None admin] PATH_INFO: `/v3/users/8528e7e46ad1478ebf804b799c72bf50` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:22.963996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b9013e-c50b-44f4-9dbe-aaf90d993006 None admin] RBAC: Authorizing `identity:get_user(user_id=8528e7e46ad1478ebf804b799c72bf50)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:22.965511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b9013e-c50b-44f4-9dbe-aaf90d993006 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:22.967452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 161/333] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:22 2022] GET /identity/v3/users/8528e7e46ad1478ebf804b799c72bf50 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:22.972943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fe813-c85a-47f7-b933-6fde37e70a40 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:23.083284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fe813-c85a-47f7-b933-6fde37e70a40 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:23.122655 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2645e8-0800-4047-bbd8-1d18b63d6153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:23.123333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2645e8-0800-4047-bbd8-1d18b63d6153 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:23.123616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2645e8-0800-4047-bbd8-1d18b63d6153 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:23.123894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2645e8-0800-4047-bbd8-1d18b63d6153 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:23.124695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2645e8-0800-4047-bbd8-1d18b63d6153 None admin] RBAC: Adding query filter params (role.id=f66cbbcc2491438184f04522c2a68578, user.id=8528e7e46ad1478ebf804b799c72bf50, scope.system=all) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:23.125005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2645e8-0800-4047-bbd8-1d18b63d6153 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:23.126452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2645e8-0800-4047-bbd8-1d18b63d6153 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:23.141153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 173/334] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Jul 28 03:46:22 2022] GET /identity/v3/role_assignments?user.id=8528e7e46ad1478ebf804b799c72bf50&scope.system=all&role.id=f66cbbcc2491438184f04522c2a68578 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:24.344834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aedf2f-cd41-4ae6-a09d-677f7eaaaa51 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:24.345358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aedf2f-cd41-4ae6-a09d-677f7eaaaa51 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:24.345677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aedf2f-cd41-4ae6-a09d-677f7eaaaa51 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:24.347179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 162/335] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:24.355781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af0c65-1e43-42d8-bcbe-873352d8cebb None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:24.356303 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af0c65-1e43-42d8-bcbe-873352d8cebb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:24.356452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af0c65-1e43-42d8-bcbe-873352d8cebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:24.378417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f7af0c65-1e43-42d8-bcbe-873352d8cebb None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:24.379638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f7af0c65-1e43-42d8-bcbe-873352d8cebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:24.387560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:24.467948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 174/336] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:24.496569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aedf2f-cd41-4ae6-a09d-677f7eaaaa51 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:24.592913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:24.632316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aedf2f-cd41-4ae6-a09d-677f7eaaaa51 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:24.676626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a86ae-4035-4db7-b7e2-d4e2506c17a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:24.677795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a86ae-4035-4db7-b7e2-d4e2506c17a1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:24.678272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a86ae-4035-4db7-b7e2-d4e2506c17a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:24.678737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a86ae-4035-4db7-b7e2-d4e2506c17a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:24.694737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a86ae-4035-4db7-b7e2-d4e2506c17a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c71a86ae-4035-4db7-b7e2-d4e2506c17a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:24.700003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:24.700847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:24.701623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:24.704205 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 163/337] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:24.710240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af0c65-1e43-42d8-bcbe-873352d8cebb None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:24.841468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af0c65-1e43-42d8-bcbe-873352d8cebb None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:24.881796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fad92a-6c02-4c41-88c1-2fa9704f52d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:24.882486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fad92a-6c02-4c41-88c1-2fa9704f52d7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:24.882717 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fad92a-6c02-4c41-88c1-2fa9704f52d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:24.883026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fad92a-6c02-4c41-88c1-2fa9704f52d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:24.883766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fad92a-6c02-4c41-88c1-2fa9704f52d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:24.884072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fad92a-6c02-4c41-88c1-2fa9704f52d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:24.885462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fad92a-6c02-4c41-88c1-2fa9704f52d7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:24.899190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 175/338] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:24.905572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a86ae-4035-4db7-b7e2-d4e2506c17a1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:25.058356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a86ae-4035-4db7-b7e2-d4e2506c17a1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:25.098551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:25.111576 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37fa1e-d26b-4c22-8bb6-3959483a2fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:25.112786 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37fa1e-d26b-4c22-8bb6-3959483a2fee None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:25.113476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37fa1e-d26b-4c22-8bb6-3959483a2fee None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:25.114011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37fa1e-d26b-4c22-8bb6-3959483a2fee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:25.201811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37fa1e-d26b-4c22-8bb6-3959483a2fee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:25.203238 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37fa1e-d26b-4c22-8bb6-3959483a2fee None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:25.328278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 164/339] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:25.334345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fad92a-6c02-4c41-88c1-2fa9704f52d7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:25.337752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:25.469445 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fad92a-6c02-4c41-88c1-2fa9704f52d7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:25.507849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18b9ff-cdf5-4520-834a-e37ca385245e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:25.508833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18b9ff-cdf5-4520-834a-e37ca385245e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:25.509318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18b9ff-cdf5-4520-834a-e37ca385245e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:25.509811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18b9ff-cdf5-4520-834a-e37ca385245e None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:25.511270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18b9ff-cdf5-4520-834a-e37ca385245e None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:25.512973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18b9ff-cdf5-4520-834a-e37ca385245e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:25.517178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 176/340] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:25 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:25.521908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18b9ff-cdf5-4520-834a-e37ca385245e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:25.522081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a55a7-5d1a-4a1b-a73b-b84fd404d9d6 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:25.522669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a55a7-5d1a-4a1b-a73b-b84fd404d9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:25.522669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a55a7-5d1a-4a1b-a73b-b84fd404d9d6 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:25.524086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 165/341] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:25.540425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fd52b-c02e-4fe8-a5c2-be9197fbf1fb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:25.541177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fd52b-c02e-4fe8-a5c2-be9197fbf1fb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:25.541177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fd52b-c02e-4fe8-a5c2-be9197fbf1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:25.576047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8f5fd52b-c02e-4fe8-a5c2-be9197fbf1fb None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:25.577271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8f5fd52b-c02e-4fe8-a5c2-be9197fbf1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:25.611139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:25.660925 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 166/342] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:25.673718 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:25.690837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bc056-271d-44ab-ba97-0cb7d5290312 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:25.691378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bc056-271d-44ab-ba97-0cb7d5290312 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:25.691800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bc056-271d-44ab-ba97-0cb7d5290312 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:25.695796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 167/343] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:46:25.700081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f57a3-d025-461e-9ae3-3c2f2340f482 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:25.700544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f57a3-d025-461e-9ae3-3c2f2340f482 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:25.700986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f57a3-d025-461e-9ae3-3c2f2340f482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:25.732498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7f1f57a3-d025-461e-9ae3-3c2f2340f482 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:25.733626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7f1f57a3-d025-461e-9ae3-3c2f2340f482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:25.740509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18b9ff-cdf5-4520-834a-e37ca385245e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:25.807976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048681da-99f0-4387-8114-671f1cd9a3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:25.808626 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048681da-99f0-4387-8114-671f1cd9a3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:25.808904 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048681da-99f0-4387-8114-671f1cd9a3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:25.809176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048681da-99f0-4387-8114-671f1cd9a3a8 None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:25.824548 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048681da-99f0-4387-8114-671f1cd9a3a8 None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:25.826009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048681da-99f0-4387-8114-671f1cd9a3a8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:25.827737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 177/344] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:25 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:25.828511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048681da-99f0-4387-8114-671f1cd9a3a8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:25.869869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 168/345] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:25.870727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f57a3-d025-461e-9ae3-3c2f2340f482 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:25.973226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048681da-99f0-4387-8114-671f1cd9a3a8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:25.994275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:26.010049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f57a3-d025-461e-9ae3-3c2f2340f482 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:26.017754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f133d-4c8f-4386-94ed-b0fe6c38a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:26.018588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f133d-4c8f-4386-94ed-b0fe6c38a2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.019443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f133d-4c8f-4386-94ed-b0fe6c38a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.019882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f133d-4c8f-4386-94ed-b0fe6c38a2d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.025886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:26.040749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f133d-4c8f-4386-94ed-b0fe6c38a2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e06f133d-4c8f-4386-94ed-b0fe6c38a2d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:26.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:26.047184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:26.048065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:26.049990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 178/346] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:26.051376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86347c-455f-4406-be6c-77e613cc1ace None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.051720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86347c-455f-4406-be6c-77e613cc1ace None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.052005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86347c-455f-4406-be6c-77e613cc1ace None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.054384 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 179/347] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:26.055972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8627e4-46fe-44e6-af30-ac2fd8934281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:26.056602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8627e4-46fe-44e6-af30-ac2fd8934281 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.056881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8627e4-46fe-44e6-af30-ac2fd8934281 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.057151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8627e4-46fe-44e6-af30-ac2fd8934281 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.057946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8627e4-46fe-44e6-af30-ac2fd8934281 None admin] RBAC: Adding query filter params (group.id=e58dbbfbc6ac4b338fc398aa98749792, role.id=f66cbbcc2491438184f04522c2a68578, scope.project.id=a75d68fb290c4a1494243d8817754029) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:26.058045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86347c-455f-4406-be6c-77e613cc1ace None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:26.058255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8627e4-46fe-44e6-af30-ac2fd8934281 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:26.061358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8627e4-46fe-44e6-af30-ac2fd8934281 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:26.069974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 169/348] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jul 28 03:46:25 2022] GET /identity/v3/role_assignments?group.id=e58dbbfbc6ac4b338fc398aa98749792&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=f66cbbcc2491438184f04522c2a68578 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:26.071325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801afc79-efd6-48b4-a86b-e8f1dcd61e45 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.071612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801afc79-efd6-48b4-a86b-e8f1dcd61e45 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.071890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801afc79-efd6-48b4-a86b-e8f1dcd61e45 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.073175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 170/349] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:26.074316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0281fc2-ea18-4966-8e17-405b049e6c9c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.074599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0281fc2-ea18-4966-8e17-405b049e6c9c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.074893 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0281fc2-ea18-4966-8e17-405b049e6c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.097979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f0281fc2-ea18-4966-8e17-405b049e6c9c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:26.100115 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f0281fc2-ea18-4966-8e17-405b049e6c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:26.179671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 171/350] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:26.180773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0281fc2-ea18-4966-8e17-405b049e6c9c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:26.195310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86347c-455f-4406-be6c-77e613cc1ace None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:26.235216 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89da6f-ea09-4767-bd7b-7529505633b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:26.236227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89da6f-ea09-4767-bd7b-7529505633b2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.236773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89da6f-ea09-4767-bd7b-7529505633b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.237335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89da6f-ea09-4767-bd7b-7529505633b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.253050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89da6f-ea09-4767-bd7b-7529505633b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-ce89da6f-ea09-4767-bd7b-7529505633b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:26.260239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:26.261293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:26.262080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:26.264627 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 180/351] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:46:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:26.266271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5223dd-b1e3-4451-b23b-916fe3d62146 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.266765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5223dd-b1e3-4451-b23b-916fe3d62146 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.267260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5223dd-b1e3-4451-b23b-916fe3d62146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.276757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:26.289285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6f5223dd-b1e3-4451-b23b-916fe3d62146 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:26.290875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6f5223dd-b1e3-4451-b23b-916fe3d62146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:26.294649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0281fc2-ea18-4966-8e17-405b049e6c9c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:26.342694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0b34d-bc71-4a25-9378-44f17c69de8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:26.344050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0b34d-bc71-4a25-9378-44f17c69de8c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.344532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0b34d-bc71-4a25-9378-44f17c69de8c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.345303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0b34d-bc71-4a25-9378-44f17c69de8c None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.346510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0b34d-bc71-4a25-9378-44f17c69de8c None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:26.347036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0b34d-bc71-4a25-9378-44f17c69de8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:26.349724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0b34d-bc71-4a25-9378-44f17c69de8c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:26.351973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:26.360946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 172/352] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:26.362101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0b34d-bc71-4a25-9378-44f17c69de8c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:26.378845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 181/353] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:26.379737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5223dd-b1e3-4451-b23b-916fe3d62146 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:26.488421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0b34d-bc71-4a25-9378-44f17c69de8c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:26.503589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5223dd-b1e3-4451-b23b-916fe3d62146 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:26.529149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe82a5a-d377-47ca-8562-f21c3fcc94bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:26.529946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe82a5a-d377-47ca-8562-f21c3fcc94bf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.530322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe82a5a-d377-47ca-8562-f21c3fcc94bf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.530713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe82a5a-d377-47ca-8562-f21c3fcc94bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.546183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe82a5a-d377-47ca-8562-f21c3fcc94bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:26.549368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b26eb0-f542-43b4-8f67-707eebc770ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:26.550147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b26eb0-f542-43b4-8f67-707eebc770ac None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.550568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b26eb0-f542-43b4-8f67-707eebc770ac None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.551027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b26eb0-f542-43b4-8f67-707eebc770ac None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.551892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b26eb0-f542-43b4-8f67-707eebc770ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:26.552890 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b26eb0-f542-43b4-8f67-707eebc770ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-efe82a5a-d377-47ca-8562-f21c3fcc94bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:26.553088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:26.554332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 03:46:26.555193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:26.556577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 173/354] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:26.557459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe82a5a-d377-47ca-8562-f21c3fcc94bf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:26.558008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b26eb0-f542-43b4-8f67-707eebc770ac None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:26.567149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:26.569734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 182/355] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:26.571276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b26eb0-f542-43b4-8f67-707eebc770ac None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:26.629546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:26.686000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe82a5a-d377-47ca-8562-f21c3fcc94bf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:26.698488 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b26eb0-f542-43b4-8f67-707eebc770ac None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:26.723773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86502ca9-e129-4b39-a767-52f76d0558cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:26.724641 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86502ca9-e129-4b39-a767-52f76d0558cf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.724928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86502ca9-e129-4b39-a767-52f76d0558cf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.725637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86502ca9-e129-4b39-a767-52f76d0558cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.741468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d19d4b-04e4-4dde-8ea0-a2b3f532b12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:26.742455 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19d4b-04e4-4dde-8ea0-a2b3f532b12f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.742983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19d4b-04e4-4dde-8ea0-a2b3f532b12f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.743480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19d4b-04e4-4dde-8ea0-a2b3f532b12f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.759240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d19d4b-04e4-4dde-8ea0-a2b3f532b12f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-f9d19d4b-04e4-4dde-8ea0-a2b3f532b12f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:26.766450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:26.767541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:26.768326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:26.771626 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 183/356] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:46:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:26.772892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d19d4b-04e4-4dde-8ea0-a2b3f532b12f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:26.822305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86502ca9-e129-4b39-a767-52f76d0558cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:26.823798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86502ca9-e129-4b39-a767-52f76d0558cf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:26.833445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:26.907579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d19d4b-04e4-4dde-8ea0-a2b3f532b12f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:26.915606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:26.950734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76278a7f-ca3f-45d5-a1a7-b230c6353482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:26.951444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278a7f-ca3f-45d5-a1a7-b230c6353482 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:26.951725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278a7f-ca3f-45d5-a1a7-b230c6353482 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:26.952133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 174/357] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:26.952259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278a7f-ca3f-45d5-a1a7-b230c6353482 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:26.952771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86502ca9-e129-4b39-a767-52f76d0558cf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:26.952874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76278a7f-ca3f-45d5-a1a7-b230c6353482 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:26.953124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76278a7f-ca3f-45d5-a1a7-b230c6353482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:26.954631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76278a7f-ca3f-45d5-a1a7-b230c6353482 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:26.966734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 184/358] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:26.967506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76278a7f-ca3f-45d5-a1a7-b230c6353482 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:27.089881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86502ca9-e129-4b39-a767-52f76d0558cf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:27.103673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76278a7f-ca3f-45d5-a1a7-b230c6353482 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:27.111174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:27.137435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d920c1-94cb-4c53-a5be-16a330a06551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:27.138056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d920c1-94cb-4c53-a5be-16a330a06551 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:27.138340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d920c1-94cb-4c53-a5be-16a330a06551 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:27.138612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d920c1-94cb-4c53-a5be-16a330a06551 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:27.150279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987b162-c4b3-4358-9eb8-4bbf1bff1fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:27.150932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b162-c4b3-4358-9eb8-4bbf1bff1fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:27.151216 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b162-c4b3-4358-9eb8-4bbf1bff1fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:27.151491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b162-c4b3-4358-9eb8-4bbf1bff1fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:27.152214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987b162-c4b3-4358-9eb8-4bbf1bff1fb7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:27.152520 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987b162-c4b3-4358-9eb8-4bbf1bff1fb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:27.153894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987b162-c4b3-4358-9eb8-4bbf1bff1fb7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:27.164680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 185/359] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:27.165925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59661f-7682-427f-b650-52fc16f03a65 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:27.166211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59661f-7682-427f-b650-52fc16f03a65 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:27.166547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59661f-7682-427f-b650-52fc16f03a65 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:27.169382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 186/360] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:27.169382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59661f-7682-427f-b650-52fc16f03a65 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:27.205499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:27.223271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d920c1-94cb-4c53-a5be-16a330a06551 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:27.226178 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d920c1-94cb-4c53-a5be-16a330a06551 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:27.292072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59661f-7682-427f-b650-52fc16f03a65 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:27.335529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74df2c-882e-49e3-b332-751585551752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:27.336179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74df2c-882e-49e3-b332-751585551752 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:27.336461 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74df2c-882e-49e3-b332-751585551752 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:27.336732 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74df2c-882e-49e3-b332-751585551752 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:27.338887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74df2c-882e-49e3-b332-751585551752 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:27.340385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74df2c-882e-49e3-b332-751585551752 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:27.342032 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 187/361] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:27 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:27.342825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74df2c-882e-49e3-b332-751585551752 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:27.354109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 175/362] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:27.354109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d920c1-94cb-4c53-a5be-16a330a06551 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:27.362348 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:27.480034 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:27.484014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74df2c-882e-49e3-b332-751585551752 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:27.504946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d920c1-94cb-4c53-a5be-16a330a06551 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:27.535503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebcf59-fb55-470d-8463-cd7d18995a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:27.536438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebcf59-fb55-470d-8463-cd7d18995a02 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:27.536438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebcf59-fb55-470d-8463-cd7d18995a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:27.537622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebcf59-fb55-470d-8463-cd7d18995a02 None admin] PATH_INFO: `/v3/users/3f5d8dd7849445c6bd56700e0ad156c3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:27.546817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebcf59-fb55-470d-8463-cd7d18995a02 None admin] RBAC: Authorizing `identity:get_user(user_id=3f5d8dd7849445c6bd56700e0ad156c3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:27.546817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebcf59-fb55-470d-8463-cd7d18995a02 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:27.547018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 188/363] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:46:27 2022] GET /identity/v3/users/3f5d8dd7849445c6bd56700e0ad156c3 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:27.548219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dff81-c54a-4d64-aa86-e815922c87d1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:27.548514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dff81-c54a-4d64-aa86-e815922c87d1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:27.548797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dff81-c54a-4d64-aa86-e815922c87d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:27.552062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc92e06-06c5-4b57-9a53-0ef6beb9db76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:27.553766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc92e06-06c5-4b57-9a53-0ef6beb9db76 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:27.554052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc92e06-06c5-4b57-9a53-0ef6beb9db76 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:27.554326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc92e06-06c5-4b57-9a53-0ef6beb9db76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:27.575014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1c9dff81-c54a-4d64-aa86-e815922c87d1 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:27.576230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1c9dff81-c54a-4d64-aa86-e815922c87d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:27.642281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc92e06-06c5-4b57-9a53-0ef6beb9db76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:27.644752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc92e06-06c5-4b57-9a53-0ef6beb9db76 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:27.654639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:27.663779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 189/364] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:27.664207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dff81-c54a-4d64-aa86-e815922c87d1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:27.788068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:27.792092 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 176/365] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:27.793208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc92e06-06c5-4b57-9a53-0ef6beb9db76 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:27.798259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dff81-c54a-4d64-aa86-e815922c87d1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:27.853542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bbb2f-dcf6-4c38-9fd0-b669f36b418f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:27.854200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bbb2f-dcf6-4c38-9fd0-b669f36b418f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:27.854482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bbb2f-dcf6-4c38-9fd0-b669f36b418f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:27.854757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bbb2f-dcf6-4c38-9fd0-b669f36b418f None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:27.856778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bbb2f-dcf6-4c38-9fd0-b669f36b418f None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:27.858231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bbb2f-dcf6-4c38-9fd0-b669f36b418f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:27.860169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 190/366] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:27 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:27.860950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bbb2f-dcf6-4c38-9fd0-b669f36b418f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:27.936387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc92e06-06c5-4b57-9a53-0ef6beb9db76 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:27.947654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:27.978773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832574a-18ce-4894-ba17-6522ff5ba266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:27.979731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832574a-18ce-4894-ba17-6522ff5ba266 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:27.980152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832574a-18ce-4894-ba17-6522ff5ba266 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:27.980563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832574a-18ce-4894-ba17-6522ff5ba266 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:27.982657 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832574a-18ce-4894-ba17-6522ff5ba266 None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:27.984363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832574a-18ce-4894-ba17-6522ff5ba266 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:27.986201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 177/367] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:27 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:27.987201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832574a-18ce-4894-ba17-6522ff5ba266 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:28.005300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bbb2f-dcf6-4c38-9fd0-b669f36b418f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:28.053679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898e0ba-b1ec-475a-bbe0-ffc2dc837124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:28.055036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898e0ba-b1ec-475a-bbe0-ffc2dc837124 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:28.055479 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898e0ba-b1ec-475a-bbe0-ffc2dc837124 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:28.055890 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898e0ba-b1ec-475a-bbe0-ffc2dc837124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:28.056823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898e0ba-b1ec-475a-bbe0-ffc2dc837124 None admin] RBAC: Adding query filter params (role.id=77b5c5dec7334e27a8126a96b7b17f32, user.id=3f5d8dd7849445c6bd56700e0ad156c3, scope.system=all) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:28.057766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898e0ba-b1ec-475a-bbe0-ffc2dc837124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:28.059755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898e0ba-b1ec-475a-bbe0-ffc2dc837124 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:28.077884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 191/368] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Jul 28 03:46:27 2022] GET /identity/v3/role_assignments?user.id=3f5d8dd7849445c6bd56700e0ad156c3&scope.system=all&role.id=77b5c5dec7334e27a8126a96b7b17f32 => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:28.077884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898e0ba-b1ec-475a-bbe0-ffc2dc837124 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:28.120518 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:28.136714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832574a-18ce-4894-ba17-6522ff5ba266 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:28.188360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4699ce7-36b3-471a-83e5-bccbd698aff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:28.189685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4699ce7-36b3-471a-83e5-bccbd698aff9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:28.190141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4699ce7-36b3-471a-83e5-bccbd698aff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:28.190585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4699ce7-36b3-471a-83e5-bccbd698aff9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:28.207882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4699ce7-36b3-471a-83e5-bccbd698aff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f4699ce7-36b3-471a-83e5-bccbd698aff9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:28.215794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:28.216747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:28.217589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:28.220422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 178/369] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:46:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:28.221455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4699ce7-36b3-471a-83e5-bccbd698aff9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:28.230745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898e0ba-b1ec-475a-bbe0-ffc2dc837124 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:28.266750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:28.280597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a18ad-35ce-4490-8935-783e427c539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:28.281475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a18ad-35ce-4490-8935-783e427c539f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:28.282423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a18ad-35ce-4490-8935-783e427c539f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:28.282423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a18ad-35ce-4490-8935-783e427c539f None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:28.283805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a18ad-35ce-4490-8935-783e427c539f None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:28.285804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a18ad-35ce-4490-8935-783e427c539f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:28.288188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 192/370] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:28 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:28.288969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a18ad-35ce-4490-8935-783e427c539f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:28.358215 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4699ce7-36b3-471a-83e5-bccbd698aff9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:28.396955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518bbe2-8094-4f75-b135-0a5b1fc3a435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:28.397724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518bbe2-8094-4f75-b135-0a5b1fc3a435 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:28.398006 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518bbe2-8094-4f75-b135-0a5b1fc3a435 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:28.398277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518bbe2-8094-4f75-b135-0a5b1fc3a435 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:28.399101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518bbe2-8094-4f75-b135-0a5b1fc3a435 None admin] RBAC: Adding query filter params (group.id=e58dbbfbc6ac4b338fc398aa98749792, role.id=f66cbbcc2491438184f04522c2a68578, scope.project.id=6409fa7c592e4bd09561cddf768267d4) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:28.399409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518bbe2-8094-4f75-b135-0a5b1fc3a435 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:28.401025 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518bbe2-8094-4f75-b135-0a5b1fc3a435 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:28.404571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:28.408667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 179/371] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jul 28 03:46:28 2022] GET /identity/v3/role_assignments?group.id=e58dbbfbc6ac4b338fc398aa98749792&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=f66cbbcc2491438184f04522c2a68578 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:28.409215 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518bbe2-8094-4f75-b135-0a5b1fc3a435 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:28.417028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a18ad-35ce-4490-8935-783e427c539f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:28.470744 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad92df2-4e01-47fd-b12e-d9dd2b34fb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:28.471442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad92df2-4e01-47fd-b12e-d9dd2b34fb9b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:28.471714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad92df2-4e01-47fd-b12e-d9dd2b34fb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:28.471989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad92df2-4e01-47fd-b12e-d9dd2b34fb9b None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:28.473274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad92df2-4e01-47fd-b12e-d9dd2b34fb9b None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:28.476080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad92df2-4e01-47fd-b12e-d9dd2b34fb9b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:28.478871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 193/372] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:28 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:28.480010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad92df2-4e01-47fd-b12e-d9dd2b34fb9b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:28.544752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:28.549393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518bbe2-8094-4f75-b135-0a5b1fc3a435 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:28.595961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e01d29-1ad6-485e-aeb0-15c495f31c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:28.597242 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e01d29-1ad6-485e-aeb0-15c495f31c47 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:28.597670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e01d29-1ad6-485e-aeb0-15c495f31c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:28.598112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e01d29-1ad6-485e-aeb0-15c495f31c47 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:28.599030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e01d29-1ad6-485e-aeb0-15c495f31c47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:28.600110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e01d29-1ad6-485e-aeb0-15c495f31c47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:28.601927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e01d29-1ad6-485e-aeb0-15c495f31c47 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:28.610774 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 180/373] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:28.613422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefe747-7203-40ba-a229-f6306821c393 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:28.613854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefe747-7203-40ba-a229-f6306821c393 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:28.614273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefe747-7203-40ba-a229-f6306821c393 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:28.616469 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 181/374] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:28.617481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefe747-7203-40ba-a229-f6306821c393 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:28.621789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad92df2-4e01-47fd-b12e-d9dd2b34fb9b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:28.684328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d70d5-41eb-4e58-89c8-79c149537b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:28.684994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d70d5-41eb-4e58-89c8-79c149537b99 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:28.685283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d70d5-41eb-4e58-89c8-79c149537b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:28.685567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d70d5-41eb-4e58-89c8-79c149537b99 None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:28.686767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d70d5-41eb-4e58-89c8-79c149537b99 None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:28.688337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d70d5-41eb-4e58-89c8-79c149537b99 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:28.690261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 194/375] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:28 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:28.691151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d70d5-41eb-4e58-89c8-79c149537b99 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:28.720078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:28.767860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefe747-7203-40ba-a229-f6306821c393 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:28.814013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb2b85-bbc0-41e5-9983-d39b9f0510f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:28.814660 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb2b85-bbc0-41e5-9983-d39b9f0510f4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:28.815009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb2b85-bbc0-41e5-9983-d39b9f0510f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:28.815285 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb2b85-bbc0-41e5-9983-d39b9f0510f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:28.816071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb2b85-bbc0-41e5-9983-d39b9f0510f4 None admin] RBAC: Adding query filter params (group.id=e58dbbfbc6ac4b338fc398aa98749792, role.id=854a7f7c663b4c66a08f7e731c7bf63c, scope.project.id=a75d68fb290c4a1494243d8817754029) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:28.816380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb2b85-bbc0-41e5-9983-d39b9f0510f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:28.817870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb2b85-bbc0-41e5-9983-d39b9f0510f4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:28.824198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 182/376] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jul 28 03:46:28 2022] GET /identity/v3/role_assignments?group.id=e58dbbfbc6ac4b338fc398aa98749792&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=854a7f7c663b4c66a08f7e731c7bf63c => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:28.825419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd41a31-9891-45f4-8227-2eb104abbc30 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:28.825720 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd41a31-9891-45f4-8227-2eb104abbc30 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:28.825994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd41a31-9891-45f4-8227-2eb104abbc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:28.829540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d70d5-41eb-4e58-89c8-79c149537b99 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:28.847210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ebd41a31-9891-45f4-8227-2eb104abbc30 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:28.848688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ebd41a31-9891-45f4-8227-2eb104abbc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:28.871053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:28.874183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94854ef3-00e9-4be6-885a-f1b3a2626bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:28.875101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94854ef3-00e9-4be6-885a-f1b3a2626bdb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:28.875562 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94854ef3-00e9-4be6-885a-f1b3a2626bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:28.876010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94854ef3-00e9-4be6-885a-f1b3a2626bdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:28.929578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 183/377] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:28.931159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd41a31-9891-45f4-8227-2eb104abbc30 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:28.962345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94854ef3-00e9-4be6-885a-f1b3a2626bdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:28.964407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94854ef3-00e9-4be6-885a-f1b3a2626bdb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:29.008574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:29.056443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd41a31-9891-45f4-8227-2eb104abbc30 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:29.094448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce5c75-4a8d-48d7-b0ab-a84e2da005ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:29.095099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce5c75-4a8d-48d7-b0ab-a84e2da005ba None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:29.095384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce5c75-4a8d-48d7-b0ab-a84e2da005ba None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:29.095658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce5c75-4a8d-48d7-b0ab-a84e2da005ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:29.096449 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce5c75-4a8d-48d7-b0ab-a84e2da005ba None admin] RBAC: Adding query filter params (group.id=e58dbbfbc6ac4b338fc398aa98749792, role.id=854a7f7c663b4c66a08f7e731c7bf63c, scope.project.id=6409fa7c592e4bd09561cddf768267d4) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:29.096760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce5c75-4a8d-48d7-b0ab-a84e2da005ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:29.099734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce5c75-4a8d-48d7-b0ab-a84e2da005ba None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:29.102065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 195/378] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:29.103366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94854ef3-00e9-4be6-885a-f1b3a2626bdb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:29.108810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 184/379] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jul 28 03:46:28 2022] GET /identity/v3/role_assignments?group.id=e58dbbfbc6ac4b338fc398aa98749792&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=854a7f7c663b4c66a08f7e731c7bf63c => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:29.109726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce5c75-4a8d-48d7-b0ab-a84e2da005ba None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:29.134287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:29.229408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94854ef3-00e9-4be6-885a-f1b3a2626bdb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:29.248999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce5c75-4a8d-48d7-b0ab-a84e2da005ba None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:29.267077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:29.282414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2e113-fb90-42ef-ad48-4633199519e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:29.284196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2e113-fb90-42ef-ad48-4633199519e0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:29.285209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2e113-fb90-42ef-ad48-4633199519e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:29.285500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2e113-fb90-42ef-ad48-4633199519e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:29.305027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fe677-bf03-40df-9553-c00f6c51e5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:29.305685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fe677-bf03-40df-9553-c00f6c51e5df None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:29.305968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fe677-bf03-40df-9553-c00f6c51e5df None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:29.306245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fe677-bf03-40df-9553-c00f6c51e5df None admin] PATH_INFO: `/v3/projects/826fca2fab6740f6870c6426848e8d90` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:29.310091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fe677-bf03-40df-9553-c00f6c51e5df None admin] RBAC: Authorizing `identity:get_project(project_id=826fca2fab6740f6870c6426848e8d90)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:29.311856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fe677-bf03-40df-9553-c00f6c51e5df None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:29.314302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2e113-fb90-42ef-ad48-4633199519e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:29.318616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 185/380] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:29 2022] GET /identity/v3/projects/826fca2fab6740f6870c6426848e8d90 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:29.320133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fe677-bf03-40df-9553-c00f6c51e5df None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-69f2e113-fb90-42ef-ad48-4633199519e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:29.323385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:29.324332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:29.325161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:29.327047 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 196/381] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:29.331840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2e113-fb90-42ef-ad48-4633199519e0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:29.485537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:29.502594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fe677-bf03-40df-9553-c00f6c51e5df None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:29.542812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2e113-fb90-42ef-ad48-4633199519e0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:29.543150 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b15be-74fb-4976-8f91-bc3dda6e94e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:29.543808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b15be-74fb-4976-8f91-bc3dda6e94e0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:29.544091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b15be-74fb-4976-8f91-bc3dda6e94e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:29.544369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b15be-74fb-4976-8f91-bc3dda6e94e0 None admin] PATH_INFO: `/v3/groups/3abac29c895647c699c4c15980e768c3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:29.557060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b15be-74fb-4976-8f91-bc3dda6e94e0 None admin] RBAC: Authorizing `identity:get_group(group_id=3abac29c895647c699c4c15980e768c3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:29.558533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b15be-74fb-4976-8f91-bc3dda6e94e0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:29.560685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 186/382] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:29 2022] GET /identity/v3/groups/3abac29c895647c699c4c15980e768c3 => generated 229 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:29.565988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b15be-74fb-4976-8f91-bc3dda6e94e0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:29.611314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66169b21-c9de-462b-b60d-036f62bd0ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:29.611936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66169b21-c9de-462b-b60d-036f62bd0ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:29.612213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66169b21-c9de-462b-b60d-036f62bd0ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:29.612482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66169b21-c9de-462b-b60d-036f62bd0ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:29.613201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66169b21-c9de-462b-b60d-036f62bd0ff0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:29.613503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66169b21-c9de-462b-b60d-036f62bd0ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:29.615423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66169b21-c9de-462b-b60d-036f62bd0ff0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:29.633907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 197/383] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:29.639527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66169b21-c9de-462b-b60d-036f62bd0ff0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:29.725931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:29.742829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b15be-74fb-4976-8f91-bc3dda6e94e0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:29.777695 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ddf7e-9364-4933-bb64-01016fa0b53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:29.778352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ddf7e-9364-4933-bb64-01016fa0b53a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:29.778635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ddf7e-9364-4933-bb64-01016fa0b53a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:29.778936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ddf7e-9364-4933-bb64-01016fa0b53a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:29.779792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ddf7e-9364-4933-bb64-01016fa0b53a None admin] RBAC: Adding query filter params (group.id=3abac29c895647c699c4c15980e768c3, role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=826fca2fab6740f6870c6426848e8d90) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:29.780098 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ddf7e-9364-4933-bb64-01016fa0b53a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:29.781616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ddf7e-9364-4933-bb64-01016fa0b53a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:29.787749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 187/384] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jul 28 03:46:29 2022] GET /identity/v3/role_assignments?group.id=3abac29c895647c699c4c15980e768c3&scope.project.id=826fca2fab6740f6870c6426848e8d90&role.id=a37ae24c516b48c69e13721c013134a2 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:29.825394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66169b21-c9de-462b-b60d-036f62bd0ff0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:29.873861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f574f1a-fe06-49ab-8676-33b2ea8c53b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:29.874511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f574f1a-fe06-49ab-8676-33b2ea8c53b8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:29.874809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f574f1a-fe06-49ab-8676-33b2ea8c53b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:29.875100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f574f1a-fe06-49ab-8676-33b2ea8c53b8 None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:29.876256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f574f1a-fe06-49ab-8676-33b2ea8c53b8 None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:29.877761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f574f1a-fe06-49ab-8676-33b2ea8c53b8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:29.879692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 198/385] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:29 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:29.887906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ddf7e-9364-4933-bb64-01016fa0b53a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:29.915870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:30.007582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ddf7e-9364-4933-bb64-01016fa0b53a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:30.041352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacb04d-f290-4155-b5b4-b3bce911aafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:30.041989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacb04d-f290-4155-b5b4-b3bce911aafd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:30.042270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacb04d-f290-4155-b5b4-b3bce911aafd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:30.042543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacb04d-f290-4155-b5b4-b3bce911aafd None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:30.043741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacb04d-f290-4155-b5b4-b3bce911aafd None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:30.045169 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacb04d-f290-4155-b5b4-b3bce911aafd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:30.047140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 188/386] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:29 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:30.053762 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f574f1a-fe06-49ab-8676-33b2ea8c53b8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:30.214692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f574f1a-fe06-49ab-8676-33b2ea8c53b8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:30.241090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:30.269854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e13182-3df8-4ddc-ab6e-09abfa7417ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:30.270544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e13182-3df8-4ddc-ab6e-09abfa7417ef None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:30.270907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e13182-3df8-4ddc-ab6e-09abfa7417ef None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:30.271341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e13182-3df8-4ddc-ab6e-09abfa7417ef None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029/groups/e58dbbfbc6ac4b338fc398aa98749792/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:30.279555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e13182-3df8-4ddc-ab6e-09abfa7417ef None admin] RBAC: Authorizing `identity:create_grant(project_id=a75d68fb290c4a1494243d8817754029, group_id=e58dbbfbc6ac4b338fc398aa98749792, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:30.281041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e13182-3df8-4ddc-ab6e-09abfa7417ef None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:30.307609 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 199/387] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Jul 28 03:46:30 2022] PUT /identity/v3/projects/a75d68fb290c4a1494243d8817754029/groups/e58dbbfbc6ac4b338fc398aa98749792/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:30.711802 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683604b6-c171-42a0-b612-9c3b706ae17a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:30.712214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683604b6-c171-42a0-b612-9c3b706ae17a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:30.712397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683604b6-c171-42a0-b612-9c3b706ae17a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:30.713987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 189/388] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:30.719343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcec51-d805-4266-9812-d9ae07fab25d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:30.719834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcec51-d805-4266-9812-d9ae07fab25d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:30.719911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcec51-d805-4266-9812-d9ae07fab25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:30.739289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f0bcec51-d805-4266-9812-d9ae07fab25d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:30.741619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f0bcec51-d805-4266-9812-d9ae07fab25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:30.836138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 200/389] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:30.870137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683604b6-c171-42a0-b612-9c3b706ae17a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:30.998435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:31.023027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683604b6-c171-42a0-b612-9c3b706ae17a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:31.073354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c1dcd-1a82-402f-b06a-b5831a24477b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:31.074362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c1dcd-1a82-402f-b06a-b5831a24477b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:31.074879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c1dcd-1a82-402f-b06a-b5831a24477b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:31.075345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c1dcd-1a82-402f-b06a-b5831a24477b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:31.086233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c1dcd-1a82-402f-b06a-b5831a24477b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-594c1dcd-1a82-402f-b06a-b5831a24477b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:31.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:31.092862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:31.093642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:31.096197 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 190/390] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:31.102078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcec51-d805-4266-9812-d9ae07fab25d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:31.238701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcec51-d805-4266-9812-d9ae07fab25d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:31.249575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:31.286262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5db10-a125-4f7b-b71d-c20bf95067ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:31.287230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5db10-a125-4f7b-b71d-c20bf95067ac None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:31.287591 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5db10-a125-4f7b-b71d-c20bf95067ac None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:31.287873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5db10-a125-4f7b-b71d-c20bf95067ac None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:31.288595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5db10-a125-4f7b-b71d-c20bf95067ac None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:31.288897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5db10-a125-4f7b-b71d-c20bf95067ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:31.290281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5db10-a125-4f7b-b71d-c20bf95067ac None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:31.303658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 201/391] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:31.309118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c1dcd-1a82-402f-b06a-b5831a24477b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:31.366382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadee73-e53f-402f-ab56-d049ca743e19 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:31.366671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadee73-e53f-402f-ab56-d049ca743e19 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:31.366881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadee73-e53f-402f-ab56-d049ca743e19 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:31.368400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 202/392] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:46:31.373361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc715eb5-9811-4f2d-b4bb-1edec8be4360 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:31.373641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc715eb5-9811-4f2d-b4bb-1edec8be4360 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:31.373913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc715eb5-9811-4f2d-b4bb-1edec8be4360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:31.397658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cc715eb5-9811-4f2d-b4bb-1edec8be4360 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:31.398857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cc715eb5-9811-4f2d-b4bb-1edec8be4360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:31.482919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c1dcd-1a82-402f-b06a-b5831a24477b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:31.524326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 203/393] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:31.534535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af7223-c072-43c5-92ff-14a2afe057f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:31.538122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af7223-c072-43c5-92ff-14a2afe057f3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:31.538122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af7223-c072-43c5-92ff-14a2afe057f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:31.538122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af7223-c072-43c5-92ff-14a2afe057f3 None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:31.538122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af7223-c072-43c5-92ff-14a2afe057f3 None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:31.538755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af7223-c072-43c5-92ff-14a2afe057f3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:31.541115 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 191/394] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:31 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:31.546089 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc715eb5-9811-4f2d-b4bb-1edec8be4360 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:31.565425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af7223-c072-43c5-92ff-14a2afe057f3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:31.587811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:31.725073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:31.726501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af7223-c072-43c5-92ff-14a2afe057f3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:31.736701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc715eb5-9811-4f2d-b4bb-1edec8be4360 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:31.766132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e3acb-288c-4f14-80d5-a2baa8e28aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:31.766924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e3acb-288c-4f14-80d5-a2baa8e28aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:31.767277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e3acb-288c-4f14-80d5-a2baa8e28aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:31.767622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e3acb-288c-4f14-80d5-a2baa8e28aa7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:31.779594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e3acb-288c-4f14-80d5-a2baa8e28aa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-fd6e3acb-288c-4f14-80d5-a2baa8e28aa7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:31.785611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:31.786886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:31.787928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 192/395] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:46:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:31.801345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e3acb-288c-4f14-80d5-a2baa8e28aa7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:31.816024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62234f3e-8ad7-47b4-b241-ed5b485c0f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:31.816888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62234f3e-8ad7-47b4-b241-ed5b485c0f30 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:31.817333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62234f3e-8ad7-47b4-b241-ed5b485c0f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:31.817812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62234f3e-8ad7-47b4-b241-ed5b485c0f30 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:31.819881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62234f3e-8ad7-47b4-b241-ed5b485c0f30 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:31.823572 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62234f3e-8ad7-47b4-b241-ed5b485c0f30 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:31.825395 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 204/396] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:31 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:31.831367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62234f3e-8ad7-47b4-b241-ed5b485c0f30 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:31.950493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62234f3e-8ad7-47b4-b241-ed5b485c0f30 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:31.956599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:31.990109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655b67a-e4d7-48a1-a69a-25cea1829c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:31.991055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655b67a-e4d7-48a1-a69a-25cea1829c6a None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:31.991482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655b67a-e4d7-48a1-a69a-25cea1829c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:31.992283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655b67a-e4d7-48a1-a69a-25cea1829c6a None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4/groups/e58dbbfbc6ac4b338fc398aa98749792/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:31.994522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655b67a-e4d7-48a1-a69a-25cea1829c6a None admin] RBAC: Authorizing `identity:create_grant(project_id=6409fa7c592e4bd09561cddf768267d4, group_id=e58dbbfbc6ac4b338fc398aa98749792, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:31.996165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655b67a-e4d7-48a1-a69a-25cea1829c6a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:32.014119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 205/397] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Jul 28 03:46:31 2022] PUT /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4/groups/e58dbbfbc6ac4b338fc398aa98749792/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:32.030423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e3acb-288c-4f14-80d5-a2baa8e28aa7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:32.043334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586bb70-9bfa-4afa-9dc4-62f6dac37c37 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.043851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586bb70-9bfa-4afa-9dc4-62f6dac37c37 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.044372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586bb70-9bfa-4afa-9dc4-62f6dac37c37 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.046329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 206/398] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:32.069156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61e817-1c31-41b5-9093-1b8624ee4956 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.069701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61e817-1c31-41b5-9093-1b8624ee4956 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.070119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61e817-1c31-41b5-9093-1b8624ee4956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.105297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac8924-02fb-4202-bc84-172c24023bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:32.107000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac8924-02fb-4202-bc84-172c24023bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.107766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac8924-02fb-4202-bc84-172c24023bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.107989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fb61e817-1c31-41b5-9093-1b8624ee4956 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:32.108889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac8924-02fb-4202-bc84-172c24023bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.109933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fb61e817-1c31-41b5-9093-1b8624ee4956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:32.111001 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac8924-02fb-4202-bc84-172c24023bd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:32.112425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac8924-02fb-4202-bc84-172c24023bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:32.114056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac8924-02fb-4202-bc84-172c24023bd6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:32.141708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 193/399] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:32.159217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:32.160114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac8924-02fb-4202-bc84-172c24023bd6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:32.241788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 207/400] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:32.268515 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61e817-1c31-41b5-9093-1b8624ee4956 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:32.301422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac8924-02fb-4202-bc84-172c24023bd6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:32.351947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:32.357808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502de1f-35b6-4d1a-b7d5-408bcde551ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:32.358658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502de1f-35b6-4d1a-b7d5-408bcde551ca None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.359078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502de1f-35b6-4d1a-b7d5-408bcde551ca None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.359457 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502de1f-35b6-4d1a-b7d5-408bcde551ca None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.360731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502de1f-35b6-4d1a-b7d5-408bcde551ca None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:32.362265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502de1f-35b6-4d1a-b7d5-408bcde551ca None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:32.364403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 194/401] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:32 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:32.365842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5b220-9293-4d8a-9b67-96d3607200f8 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.366185 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5b220-9293-4d8a-9b67-96d3607200f8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.366537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5b220-9293-4d8a-9b67-96d3607200f8 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.368031 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 195/402] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:32.372646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5b220-9293-4d8a-9b67-96d3607200f8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:32.410330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61e817-1c31-41b5-9093-1b8624ee4956 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:32.462161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610155e-2b0a-4b3c-b698-c4d431b9372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:32.463000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610155e-2b0a-4b3c-b698-c4d431b9372e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.463411 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610155e-2b0a-4b3c-b698-c4d431b9372e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.463761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610155e-2b0a-4b3c-b698-c4d431b9372e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.476087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610155e-2b0a-4b3c-b698-c4d431b9372e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-2610155e-2b0a-4b3c-b698-c4d431b9372e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:32.481980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:32.482906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:32.483682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 208/403] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:46:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:32.485253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd2293-cff2-49cb-8953-e1374894345b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.485546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd2293-cff2-49cb-8953-e1374894345b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.485906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd2293-cff2-49cb-8953-e1374894345b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.490957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5b220-9293-4d8a-9b67-96d3607200f8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:32.504972 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9ddd2293-cff2-49cb-8953-e1374894345b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:32.506761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9ddd2293-cff2-49cb-8953-e1374894345b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:32.519146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:32.527718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b46a8-dc13-4837-aeb2-b36c579a388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:32.528430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b46a8-dc13-4837-aeb2-b36c579a388c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.528747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b46a8-dc13-4837-aeb2-b36c579a388c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.529027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b46a8-dc13-4837-aeb2-b36c579a388c None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.531203 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b46a8-dc13-4837-aeb2-b36c579a388c None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:32.534520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b46a8-dc13-4837-aeb2-b36c579a388c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:32.543710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 196/404] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:32 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:32.545057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b46a8-dc13-4837-aeb2-b36c579a388c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:32.594854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 209/405] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:32.595947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd2293-cff2-49cb-8953-e1374894345b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:32.632009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:32.672437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b46a8-dc13-4837-aeb2-b36c579a388c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:32.700734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd2293-cff2-49cb-8953-e1374894345b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:32.708886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc96cdf-d3aa-4e20-96ce-aa5beb2d595b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:32.709510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96cdf-d3aa-4e20-96ce-aa5beb2d595b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.709796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96cdf-d3aa-4e20-96ce-aa5beb2d595b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.710338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96cdf-d3aa-4e20-96ce-aa5beb2d595b None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.710797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc96cdf-d3aa-4e20-96ce-aa5beb2d595b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:32.711114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc96cdf-d3aa-4e20-96ce-aa5beb2d595b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:32.712527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc96cdf-d3aa-4e20-96ce-aa5beb2d595b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:32.720985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 197/406] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:32.721977 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5ca12-a5fe-402f-83c7-81b8580a9aad None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.722277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5ca12-a5fe-402f-83c7-81b8580a9aad None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.722555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5ca12-a5fe-402f-83c7-81b8580a9aad None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.724605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 198/407] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:32.725301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5ca12-a5fe-402f-83c7-81b8580a9aad None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:32.747680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e536ed4-e795-4c33-af04-8ee86ea41ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:32.749571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e536ed4-e795-4c33-af04-8ee86ea41ccb None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.750059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e536ed4-e795-4c33-af04-8ee86ea41ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.750773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e536ed4-e795-4c33-af04-8ee86ea41ccb None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029/groups/e58dbbfbc6ac4b338fc398aa98749792/roles/854a7f7c663b4c66a08f7e731c7bf63c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.754164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e536ed4-e795-4c33-af04-8ee86ea41ccb None admin] RBAC: Authorizing `identity:create_grant(project_id=a75d68fb290c4a1494243d8817754029, group_id=e58dbbfbc6ac4b338fc398aa98749792, role_id=854a7f7c663b4c66a08f7e731c7bf63c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:32.757032 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e536ed4-e795-4c33-af04-8ee86ea41ccb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:32.774882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 210/408] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Jul 28 03:46:32 2022] PUT /identity/v3/projects/a75d68fb290c4a1494243d8817754029/groups/e58dbbfbc6ac4b338fc398aa98749792/roles/854a7f7c663b4c66a08f7e731c7bf63c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:32.775950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e536ed4-e795-4c33-af04-8ee86ea41ccb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:32.785642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:32.858699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5ca12-a5fe-402f-83c7-81b8580a9aad None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:32.894059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:32.899457 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e536ed4-e795-4c33-af04-8ee86ea41ccb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:32.900862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811f77f-f665-4eea-a437-038ae5f80ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:32.901773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811f77f-f665-4eea-a437-038ae5f80ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.902296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811f77f-f665-4eea-a437-038ae5f80ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.902746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811f77f-f665-4eea-a437-038ae5f80ee3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.917159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811f77f-f665-4eea-a437-038ae5f80ee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d811f77f-f665-4eea-a437-038ae5f80ee3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:32.923173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:32.924102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:32.924890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:32.928292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 199/409] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:46:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:32.930120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67c9e-20d0-4399-bd10-f1cf2896d00f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.930627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67c9e-20d0-4399-bd10-f1cf2896d00f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.931135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67c9e-20d0-4399-bd10-f1cf2896d00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.938453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63fde1-4019-4c57-a7f6-2c2c7a19b5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:32.939229 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63fde1-4019-4c57-a7f6-2c2c7a19b5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:32.939581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63fde1-4019-4c57-a7f6-2c2c7a19b5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:32.939928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63fde1-4019-4c57-a7f6-2c2c7a19b5a7 None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:32.941229 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63fde1-4019-4c57-a7f6-2c2c7a19b5a7 None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:32.943322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63fde1-4019-4c57-a7f6-2c2c7a19b5a7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:32.949676 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 211/410] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:32 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:32.950736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63fde1-4019-4c57-a7f6-2c2c7a19b5a7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:32.954636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-51e67c9e-20d0-4399-bd10-f1cf2896d00f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:32.956276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-51e67c9e-20d0-4399-bd10-f1cf2896d00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:33.037784 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:33.046234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 200/411] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:33.047930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e67c9e-20d0-4399-bd10-f1cf2896d00f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:33.086382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63fde1-4019-4c57-a7f6-2c2c7a19b5a7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:33.121690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c0859-500b-4fe0-9617-1ac30953cfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:33.122238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c0859-500b-4fe0-9617-1ac30953cfcc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:33.122521 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c0859-500b-4fe0-9617-1ac30953cfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:33.122817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c0859-500b-4fe0-9617-1ac30953cfcc None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:33.123543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c0859-500b-4fe0-9617-1ac30953cfcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:33.123834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c0859-500b-4fe0-9617-1ac30953cfcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:33.125197 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c0859-500b-4fe0-9617-1ac30953cfcc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:33.134968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 212/412] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:33.135793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c0859-500b-4fe0-9617-1ac30953cfcc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:33.176015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e67c9e-20d0-4399-bd10-f1cf2896d00f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:33.204194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:33.225231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5be90-c15d-44dd-afc9-ccaaa59ee611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:33.226524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5be90-c15d-44dd-afc9-ccaaa59ee611 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:33.227070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5be90-c15d-44dd-afc9-ccaaa59ee611 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:33.227727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5be90-c15d-44dd-afc9-ccaaa59ee611 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:33.229642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5be90-c15d-44dd-afc9-ccaaa59ee611 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:33.232036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5be90-c15d-44dd-afc9-ccaaa59ee611 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:33.235126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 201/413] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:33 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:33.236895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5be90-c15d-44dd-afc9-ccaaa59ee611 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:33.245375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c0859-500b-4fe0-9617-1ac30953cfcc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:33.290176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4584402-4b51-484c-8181-44b7b7eb1734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:33.291559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4584402-4b51-484c-8181-44b7b7eb1734 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:33.292406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4584402-4b51-484c-8181-44b7b7eb1734 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:33.292903 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4584402-4b51-484c-8181-44b7b7eb1734 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:33.297007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:33.309376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4584402-4b51-484c-8181-44b7b7eb1734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-b4584402-4b51-484c-8181-44b7b7eb1734 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:33.317861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:33.318735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:33.319560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:33.322916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 213/414] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:33.324212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4584402-4b51-484c-8181-44b7b7eb1734 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:33.358235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5be90-c15d-44dd-afc9-ccaaa59ee611 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:33.395048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188616e1-0ccd-4821-9c44-1e47cbc24179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:33.396575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188616e1-0ccd-4821-9c44-1e47cbc24179 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:33.396706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188616e1-0ccd-4821-9c44-1e47cbc24179 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:33.396892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188616e1-0ccd-4821-9c44-1e47cbc24179 None admin] PATH_INFO: `/v3/groups/3abac29c895647c699c4c15980e768c3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:33.398223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188616e1-0ccd-4821-9c44-1e47cbc24179 None admin] RBAC: Authorizing `identity:get_group(group_id=3abac29c895647c699c4c15980e768c3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:33.399807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188616e1-0ccd-4821-9c44-1e47cbc24179 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:33.401812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 202/415] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:33 2022] GET /identity/v3/groups/3abac29c895647c699c4c15980e768c3 => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:33.402577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188616e1-0ccd-4821-9c44-1e47cbc24179 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:33.448100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4584402-4b51-484c-8181-44b7b7eb1734 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:33.472249 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:33.495310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfffad0-3bbd-453a-b9dd-5184c5bfbdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:33.496452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfffad0-3bbd-453a-b9dd-5184c5bfbdbf None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:33.496818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfffad0-3bbd-453a-b9dd-5184c5bfbdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:33.497221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfffad0-3bbd-453a-b9dd-5184c5bfbdbf None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4/groups/e58dbbfbc6ac4b338fc398aa98749792/roles/854a7f7c663b4c66a08f7e731c7bf63c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:33.500223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfffad0-3bbd-453a-b9dd-5184c5bfbdbf None admin] RBAC: Authorizing `identity:create_grant(project_id=6409fa7c592e4bd09561cddf768267d4, group_id=e58dbbfbc6ac4b338fc398aa98749792, role_id=854a7f7c663b4c66a08f7e731c7bf63c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:33.502281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfffad0-3bbd-453a-b9dd-5184c5bfbdbf None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:33.521246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 214/416] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Jul 28 03:46:33 2022] PUT /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4/groups/e58dbbfbc6ac4b338fc398aa98749792/roles/854a7f7c663b4c66a08f7e731c7bf63c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:33.522100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188616e1-0ccd-4821-9c44-1e47cbc24179 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:33.522302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfffad0-3bbd-453a-b9dd-5184c5bfbdbf None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:33.606270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5079c8b-ac9b-4300-b1be-9170b6fe089d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:33.606928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5079c8b-ac9b-4300-b1be-9170b6fe089d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:33.607207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5079c8b-ac9b-4300-b1be-9170b6fe089d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:33.607476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5079c8b-ac9b-4300-b1be-9170b6fe089d None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:33.608343 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5079c8b-ac9b-4300-b1be-9170b6fe089d None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:33.608651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5079c8b-ac9b-4300-b1be-9170b6fe089d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:33.610054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5079c8b-ac9b-4300-b1be-9170b6fe089d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:33.626694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 203/417] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:33.633062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5079c8b-ac9b-4300-b1be-9170b6fe089d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:33.653668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:33.719551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfffad0-3bbd-453a-b9dd-5184c5bfbdbf None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:33.757126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1a248-356b-4142-8a1d-8a6743c12a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:33.757876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1a248-356b-4142-8a1d-8a6743c12a34 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:33.758116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1a248-356b-4142-8a1d-8a6743c12a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:33.758401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1a248-356b-4142-8a1d-8a6743c12a34 None admin] PATH_INFO: `/v3/projects/826fca2fab6740f6870c6426848e8d90` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:33.759724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1a248-356b-4142-8a1d-8a6743c12a34 None admin] RBAC: Authorizing `identity:get_project(project_id=826fca2fab6740f6870c6426848e8d90)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:33.761294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1a248-356b-4142-8a1d-8a6743c12a34 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:33.764579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 215/418] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:33 2022] GET /identity/v3/projects/826fca2fab6740f6870c6426848e8d90 => generated 344 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:33.771819 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1a248-356b-4142-8a1d-8a6743c12a34 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:33.778263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5079c8b-ac9b-4300-b1be-9170b6fe089d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:33.799074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:33.818376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311404e-c42a-497a-a1a4-4943c40ae3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:33.819299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311404e-c42a-497a-a1a4-4943c40ae3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:33.819578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311404e-c42a-497a-a1a4-4943c40ae3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:33.819853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311404e-c42a-497a-a1a4-4943c40ae3d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:33.932662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311404e-c42a-497a-a1a4-4943c40ae3d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:33.934321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311404e-c42a-497a-a1a4-4943c40ae3d0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:33.967140 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1a248-356b-4142-8a1d-8a6743c12a34 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:33.995229 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:34.011194 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514baf0-b1bf-4fbe-9e8f-c86ccc0e89f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:34.012274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514baf0-b1bf-4fbe-9e8f-c86ccc0e89f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:34.012765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514baf0-b1bf-4fbe-9e8f-c86ccc0e89f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:34.013218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514baf0-b1bf-4fbe-9e8f-c86ccc0e89f3 None admin] PATH_INFO: `/v3/projects/826fca2fab6740f6870c6426848e8d90/groups/3abac29c895647c699c4c15980e768c3/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:34.015685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514baf0-b1bf-4fbe-9e8f-c86ccc0e89f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=826fca2fab6740f6870c6426848e8d90, group_id=3abac29c895647c699c4c15980e768c3, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:34.017349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514baf0-b1bf-4fbe-9e8f-c86ccc0e89f3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:34.047502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 204/419] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 03:46:34.050252 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 216/420] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Jul 28 03:46:33 2022] PUT /identity/v3/projects/826fca2fab6740f6870c6426848e8d90/groups/3abac29c895647c699c4c15980e768c3/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:34.054891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311404e-c42a-497a-a1a4-4943c40ae3d0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:34.074386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:34.169581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311404e-c42a-497a-a1a4-4943c40ae3d0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:34.207562 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b55810-025c-4993-bf41-9606d56257a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:34.208131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b55810-025c-4993-bf41-9606d56257a4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:34.208635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b55810-025c-4993-bf41-9606d56257a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:34.208892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b55810-025c-4993-bf41-9606d56257a4 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:34.210121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b55810-025c-4993-bf41-9606d56257a4 None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:34.211668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b55810-025c-4993-bf41-9606d56257a4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:34.213732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 205/421] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:34 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:34.219745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514baf0-b1bf-4fbe-9e8f-c86ccc0e89f3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:34.319439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3fc66-65a5-4899-9096-1652364e4aba None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:34.319760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3fc66-65a5-4899-9096-1652364e4aba None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:34.320173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3fc66-65a5-4899-9096-1652364e4aba None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:34.321791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 206/422] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:46:34.334812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03b6e6-22d0-4da1-9877-11b64d729db3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:34.334812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03b6e6-22d0-4da1-9877-11b64d729db3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:34.334812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03b6e6-22d0-4da1-9877-11b64d729db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:34.355195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514baf0-b1bf-4fbe-9e8f-c86ccc0e89f3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:34.364148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-aa03b6e6-22d0-4da1-9877-11b64d729db3 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:34.365826 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-aa03b6e6-22d0-4da1-9877-11b64d729db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:34.392854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96251680-3fb7-4279-a679-4bcc3ad54dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:34.393881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96251680-3fb7-4279-a679-4bcc3ad54dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:34.394163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96251680-3fb7-4279-a679-4bcc3ad54dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:34.394443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96251680-3fb7-4279-a679-4bcc3ad54dc5 None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:34.395628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96251680-3fb7-4279-a679-4bcc3ad54dc5 None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:34.398354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96251680-3fb7-4279-a679-4bcc3ad54dc5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:34.401068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 217/423] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:34 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:34.406847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96251680-3fb7-4279-a679-4bcc3ad54dc5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:34.479455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 207/424] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:34.491259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:34.514038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03b6e6-22d0-4da1-9877-11b64d729db3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:34.550076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:34.570260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96251680-3fb7-4279-a679-4bcc3ad54dc5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:34.637037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a768a-e765-4245-8770-2be8ab0711cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:34.637663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a768a-e765-4245-8770-2be8ab0711cb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:34.637941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a768a-e765-4245-8770-2be8ab0711cb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:34.638214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a768a-e765-4245-8770-2be8ab0711cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:34.639030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a768a-e765-4245-8770-2be8ab0711cb None admin] RBAC: Adding query filter params (group.id=e58dbbfbc6ac4b338fc398aa98749792, role.id=f66cbbcc2491438184f04522c2a68578, scope.project.id=a75d68fb290c4a1494243d8817754029) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:34.639343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a768a-e765-4245-8770-2be8ab0711cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:34.640770 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a768a-e765-4245-8770-2be8ab0711cb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:34.647496 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 218/425] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jul 28 03:46:34 2022] GET /identity/v3/role_assignments?group.id=e58dbbfbc6ac4b338fc398aa98749792&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=f66cbbcc2491438184f04522c2a68578 => generated 611 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:34.692218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03b6e6-22d0-4da1-9877-11b64d729db3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:34.750227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0b1c1-2aa0-40cc-870f-f411749155b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:34.750820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0b1c1-2aa0-40cc-870f-f411749155b3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:34.751106 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0b1c1-2aa0-40cc-870f-f411749155b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:34.751380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0b1c1-2aa0-40cc-870f-f411749155b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:34.767859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0b1c1-2aa0-40cc-870f-f411749155b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-92a0b1c1-2aa0-40cc-870f-f411749155b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:34.773235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:34.774180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:34.774990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 208/426] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:34.784263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a768a-e765-4245-8770-2be8ab0711cb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:34.944222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a768a-e765-4245-8770-2be8ab0711cb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:34.975426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:34.992887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef3238-d272-40e3-aa02-a156a9259ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:34.993518 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef3238-d272-40e3-aa02-a156a9259ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:34.993806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef3238-d272-40e3-aa02-a156a9259ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:34.994084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef3238-d272-40e3-aa02-a156a9259ec5 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:34.994887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef3238-d272-40e3-aa02-a156a9259ec5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:34.995208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef3238-d272-40e3-aa02-a156a9259ec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:34.996600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef3238-d272-40e3-aa02-a156a9259ec5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:35.007254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 219/427] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:46:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:35.012859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0b1c1-2aa0-40cc-870f-f411749155b3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:35.113228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:35.131957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08397e5-a7b9-4865-8201-a52c5b649a34 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:35.132277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08397e5-a7b9-4865-8201-a52c5b649a34 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:35.132511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08397e5-a7b9-4865-8201-a52c5b649a34 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:35.138481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 220/428] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:35.139619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af2d2b-48cf-4763-bbca-66eaf4be224a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:35.139900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af2d2b-48cf-4763-bbca-66eaf4be224a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:35.140172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af2d2b-48cf-4763-bbca-66eaf4be224a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:35.141648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0b1c1-2aa0-40cc-870f-f411749155b3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:35.165717 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a5af2d2b-48cf-4763-bbca-66eaf4be224a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:35.166977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a5af2d2b-48cf-4763-bbca-66eaf4be224a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:35.185030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1cea6a-f368-46a6-82da-9a0eb18e4ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:35.185660 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cea6a-f368-46a6-82da-9a0eb18e4ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:35.185943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cea6a-f368-46a6-82da-9a0eb18e4ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:35.186237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cea6a-f368-46a6-82da-9a0eb18e4ee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:35.262884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 221/429] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:35.281707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1cea6a-f368-46a6-82da-9a0eb18e4ee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:35.283329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1cea6a-f368-46a6-82da-9a0eb18e4ee5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:35.286249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af2d2b-48cf-4763-bbca-66eaf4be224a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:35.374719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:35.407258 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 209/430] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:35.408314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1cea6a-f368-46a6-82da-9a0eb18e4ee5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:35.414937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af2d2b-48cf-4763-bbca-66eaf4be224a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:35.425826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:35.454313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ca85d-a3e0-40b3-ad99-bcb0d897b65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:35.455405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ca85d-a3e0-40b3-ad99-bcb0d897b65a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:35.455912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ca85d-a3e0-40b3-ad99-bcb0d897b65a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:35.456396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ca85d-a3e0-40b3-ad99-bcb0d897b65a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:35.469629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ca85d-a3e0-40b3-ad99-bcb0d897b65a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e85ca85d-a3e0-40b3-ad99-bcb0d897b65a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:35.476115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:35.476984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:35.477771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:35.485103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 222/431] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:46:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:35.486626 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ca85d-a3e0-40b3-ad99-bcb0d897b65a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:35.513466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1cea6a-f368-46a6-82da-9a0eb18e4ee5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:35.568104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab23a6-7b87-4f4c-bae4-7581dc172127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:35.568771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab23a6-7b87-4f4c-bae4-7581dc172127 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:35.569052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab23a6-7b87-4f4c-bae4-7581dc172127 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:35.569328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab23a6-7b87-4f4c-bae4-7581dc172127 None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:35.570572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab23a6-7b87-4f4c-bae4-7581dc172127 None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:35.572060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab23a6-7b87-4f4c-bae4-7581dc172127 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:35.578331 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 210/432] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:35 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:35.579384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab23a6-7b87-4f4c-bae4-7581dc172127 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:35.633167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:35.634704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ca85d-a3e0-40b3-ad99-bcb0d897b65a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:35.681691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fb28a-1287-4444-9b72-e104d2ece286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:35.682696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fb28a-1287-4444-9b72-e104d2ece286 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:35.683385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fb28a-1287-4444-9b72-e104d2ece286 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:35.683670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fb28a-1287-4444-9b72-e104d2ece286 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:35.684453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fb28a-1287-4444-9b72-e104d2ece286 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:35.684761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fb28a-1287-4444-9b72-e104d2ece286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:35.686695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fb28a-1287-4444-9b72-e104d2ece286 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:35.698500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab23a6-7b87-4f4c-bae4-7581dc172127 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:35.698745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 223/433] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:35.706459 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fb28a-1287-4444-9b72-e104d2ece286 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:35.740994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:35.744264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72d20b-4267-4dc6-8c13-d7af0b698304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:35.745050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72d20b-4267-4dc6-8c13-d7af0b698304 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:35.745429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72d20b-4267-4dc6-8c13-d7af0b698304 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:35.745803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72d20b-4267-4dc6-8c13-d7af0b698304 None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:35.747675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72d20b-4267-4dc6-8c13-d7af0b698304 None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:35.748594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72d20b-4267-4dc6-8c13-d7af0b698304 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:35.755956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 211/434] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:35 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:35.756876 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72d20b-4267-4dc6-8c13-d7af0b698304 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:35.823432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fb28a-1287-4444-9b72-e104d2ece286 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:35.866557 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedc2ae-8343-4f48-92dd-9c4d90f2499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:35.867348 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedc2ae-8343-4f48-92dd-9c4d90f2499f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:35.867726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedc2ae-8343-4f48-92dd-9c4d90f2499f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:35.868098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedc2ae-8343-4f48-92dd-9c4d90f2499f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:35.884045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72d20b-4267-4dc6-8c13-d7af0b698304 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:35.901639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:35.932427 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca84e13-cab5-4abf-b4cf-d5a0c32d003f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:35.933288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca84e13-cab5-4abf-b4cf-d5a0c32d003f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:35.933718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca84e13-cab5-4abf-b4cf-d5a0c32d003f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:35.934165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca84e13-cab5-4abf-b4cf-d5a0c32d003f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:35.935680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca84e13-cab5-4abf-b4cf-d5a0c32d003f None admin] RBAC: Adding query filter params (group.id=e58dbbfbc6ac4b338fc398aa98749792, role.id=f66cbbcc2491438184f04522c2a68578, scope.project.id=6409fa7c592e4bd09561cddf768267d4) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:35.936295 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca84e13-cab5-4abf-b4cf-d5a0c32d003f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:35.938279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca84e13-cab5-4abf-b4cf-d5a0c32d003f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:35.955044 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 212/435] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jul 28 03:46:35 2022] GET /identity/v3/role_assignments?group.id=e58dbbfbc6ac4b338fc398aa98749792&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=f66cbbcc2491438184f04522c2a68578 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:35.956673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e8664-537e-42a6-942a-a0458ad7c80f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:35.956955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e8664-537e-42a6-942a-a0458ad7c80f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:35.957232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e8664-537e-42a6-942a-a0458ad7c80f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:35.959007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 213/436] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:35.960162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedc2ae-8343-4f48-92dd-9c4d90f2499f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:35.961866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedc2ae-8343-4f48-92dd-9c4d90f2499f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:35.964328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d6e41-2272-4f73-950b-88209dc7ac5d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:35.964612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d6e41-2272-4f73-950b-88209dc7ac5d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:35.964982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d6e41-2272-4f73-950b-88209dc7ac5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:35.984391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9a7d6e41-2272-4f73-950b-88209dc7ac5d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:35.985604 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9a7d6e41-2272-4f73-950b-88209dc7ac5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:36.019180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:36.079419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 214/437] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:36.098412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d6e41-2272-4f73-950b-88209dc7ac5d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:36.102100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 224/438] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:36.107741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedc2ae-8343-4f48-92dd-9c4d90f2499f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:36.223475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d6e41-2272-4f73-950b-88209dc7ac5d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:36.230739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedc2ae-8343-4f48-92dd-9c4d90f2499f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:36.231950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:36.261390 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9abea9-a809-45c0-a17c-d7f548b4715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:36.262035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9abea9-a809-45c0-a17c-d7f548b4715a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:36.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:36.262306 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9abea9-a809-45c0-a17c-d7f548b4715a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:36.262581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9abea9-a809-45c0-a17c-d7f548b4715a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:36.269224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92260ce2-902b-4faa-a809-f3f780db2b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:36.269867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92260ce2-902b-4faa-a809-f3f780db2b67 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:36.270144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92260ce2-902b-4faa-a809-f3f780db2b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:36.270415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92260ce2-902b-4faa-a809-f3f780db2b67 None admin] PATH_INFO: `/v3/projects/a75d68fb290c4a1494243d8817754029` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:36.273001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92260ce2-902b-4faa-a809-f3f780db2b67 None admin] RBAC: Authorizing `identity:get_project(project_id=a75d68fb290c4a1494243d8817754029)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:36.274320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9abea9-a809-45c0-a17c-d7f548b4715a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:36.274425 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92260ce2-902b-4faa-a809-f3f780db2b67 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:36.276274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 225/439] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:36 2022] GET /identity/v3/projects/a75d68fb290c4a1494243d8817754029 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-df9abea9-a809-45c0-a17c-d7f548b4715a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:36.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:36.280131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:36.285959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 215/440] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:46:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:36.286981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92260ce2-902b-4faa-a809-f3f780db2b67 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:36.287065 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9abea9-a809-45c0-a17c-d7f548b4715a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:36.391286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9abea9-a809-45c0-a17c-d7f548b4715a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:36.396827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92260ce2-902b-4faa-a809-f3f780db2b67 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:36.429229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2931e-5e65-483d-90ac-f6b18c227f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:36.430085 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2931e-5e65-483d-90ac-f6b18c227f58 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:36.430362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2931e-5e65-483d-90ac-f6b18c227f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:36.430674 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2931e-5e65-483d-90ac-f6b18c227f58 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:36.431369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2931e-5e65-483d-90ac-f6b18c227f58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:36.431672 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2931e-5e65-483d-90ac-f6b18c227f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:36.433049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2931e-5e65-483d-90ac-f6b18c227f58 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:36.436576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9272936-aee0-457c-909d-1cfe82eeb188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:36.437423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9272936-aee0-457c-909d-1cfe82eeb188 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:36.437875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9272936-aee0-457c-909d-1cfe82eeb188 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:36.438329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9272936-aee0-457c-909d-1cfe82eeb188 None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:36.439590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9272936-aee0-457c-909d-1cfe82eeb188 None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:36.441062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9272936-aee0-457c-909d-1cfe82eeb188 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:36.441501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 216/441] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:36.449483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 226/442] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:36 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:36.449626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2931e-5e65-483d-90ac-f6b18c227f58 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:36.450312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9272936-aee0-457c-909d-1cfe82eeb188 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:36.475815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:36.520967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:36.552612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2931e-5e65-483d-90ac-f6b18c227f58 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:36.556052 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9272936-aee0-457c-909d-1cfe82eeb188 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:36.584679 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4bc9-705d-4c79-a6e4-160ec029e6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:36.585296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4bc9-705d-4c79-a6e4-160ec029e6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:36.585576 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4bc9-705d-4c79-a6e4-160ec029e6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:36.585848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4bc9-705d-4c79-a6e4-160ec029e6d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:36.589824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70980105-690f-4aae-85c9-34858f2a5705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:36.590422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70980105-690f-4aae-85c9-34858f2a5705 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:36.590700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70980105-690f-4aae-85c9-34858f2a5705 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:36.591009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70980105-690f-4aae-85c9-34858f2a5705 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:36.591794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70980105-690f-4aae-85c9-34858f2a5705 None admin] RBAC: Adding query filter params (group.id=e58dbbfbc6ac4b338fc398aa98749792, role.id=854a7f7c663b4c66a08f7e731c7bf63c, scope.project.id=a75d68fb290c4a1494243d8817754029) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:36.592102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70980105-690f-4aae-85c9-34858f2a5705 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:36.593531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70980105-690f-4aae-85c9-34858f2a5705 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:36.602539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 227/443] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jul 28 03:46:36 2022] GET /identity/v3/role_assignments?group.id=e58dbbfbc6ac4b338fc398aa98749792&scope.project.id=a75d68fb290c4a1494243d8817754029&role.id=854a7f7c663b4c66a08f7e731c7bf63c => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:36.602539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d954516-5931-4ff4-b38f-29cfb8aade48 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:36.602539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d954516-5931-4ff4-b38f-29cfb8aade48 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:36.602539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d954516-5931-4ff4-b38f-29cfb8aade48 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:36.602539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 228/444] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:36.608300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32926231-bfa0-4bdd-960c-6e81f2642181 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:36.608300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32926231-bfa0-4bdd-960c-6e81f2642181 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:36.608590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32926231-bfa0-4bdd-960c-6e81f2642181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:36.629650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-32926231-bfa0-4bdd-960c-6e81f2642181 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:36.631523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-32926231-bfa0-4bdd-960c-6e81f2642181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:36.667377 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4bc9-705d-4c79-a6e4-160ec029e6d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:36.669541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4bc9-705d-4c79-a6e4-160ec029e6d0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:36.704455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:36.718809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 229/445] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:36.733837 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32926231-bfa0-4bdd-960c-6e81f2642181 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:36.800927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 217/446] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:36.807260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4bc9-705d-4c79-a6e4-160ec029e6d0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:36.808952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:36.862449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32926231-bfa0-4bdd-960c-6e81f2642181 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:36.902319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33d010-a2ec-475b-8797-7de598c2989d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:36.903000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33d010-a2ec-475b-8797-7de598c2989d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:36.903290 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33d010-a2ec-475b-8797-7de598c2989d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:36.903568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33d010-a2ec-475b-8797-7de598c2989d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:36.917693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33d010-a2ec-475b-8797-7de598c2989d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1b33d010-a2ec-475b-8797-7de598c2989d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:36.922752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:36.923672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:36.924458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 230/447] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:46:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:36.930911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33d010-a2ec-475b-8797-7de598c2989d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:36.935884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4bc9-705d-4c79-a6e4-160ec029e6d0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:36.970827 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:36.981119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ad523-cf3d-4faf-8f92-3a5f3402df3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:36.981776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ad523-cf3d-4faf-8f92-3a5f3402df3d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:36.982061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ad523-cf3d-4faf-8f92-3a5f3402df3d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:36.982339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ad523-cf3d-4faf-8f92-3a5f3402df3d None admin] PATH_INFO: `/v3/projects/6409fa7c592e4bd09561cddf768267d4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:36.983543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ad523-cf3d-4faf-8f92-3a5f3402df3d None admin] RBAC: Authorizing `identity:get_project(project_id=6409fa7c592e4bd09561cddf768267d4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:36.985002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ad523-cf3d-4faf-8f92-3a5f3402df3d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:36.986675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 218/448] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:36 2022] GET /identity/v3/projects/6409fa7c592e4bd09561cddf768267d4 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:36.992707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ad523-cf3d-4faf-8f92-3a5f3402df3d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:37.055827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33d010-a2ec-475b-8797-7de598c2989d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:37.080667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:37.101320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83382e00-456a-410d-b9ba-bc1c9d0ea546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:37.101986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83382e00-456a-410d-b9ba-bc1c9d0ea546 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:37.102271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83382e00-456a-410d-b9ba-bc1c9d0ea546 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:37.102546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83382e00-456a-410d-b9ba-bc1c9d0ea546 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:37.103294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83382e00-456a-410d-b9ba-bc1c9d0ea546 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:37.104502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83382e00-456a-410d-b9ba-bc1c9d0ea546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:37.105884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83382e00-456a-410d-b9ba-bc1c9d0ea546 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:37.107813 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ad523-cf3d-4faf-8f92-3a5f3402df3d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:37.115283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 231/449] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:37.122898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83382e00-456a-410d-b9ba-bc1c9d0ea546 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:37.147845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcd894-9e41-43ec-85d8-854a2f1436c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:37.148492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcd894-9e41-43ec-85d8-854a2f1436c7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:37.148777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcd894-9e41-43ec-85d8-854a2f1436c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:37.149051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcd894-9e41-43ec-85d8-854a2f1436c7 None admin] PATH_INFO: `/v3/groups/e58dbbfbc6ac4b338fc398aa98749792` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:37.150218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcd894-9e41-43ec-85d8-854a2f1436c7 None admin] RBAC: Authorizing `identity:get_group(group_id=e58dbbfbc6ac4b338fc398aa98749792)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:37.151688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcd894-9e41-43ec-85d8-854a2f1436c7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:37.153276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 219/450] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:36 2022] GET /identity/v3/groups/e58dbbfbc6ac4b338fc398aa98749792 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:37.159072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcd894-9e41-43ec-85d8-854a2f1436c7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:37.221105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:37.222557 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83382e00-456a-410d-b9ba-bc1c9d0ea546 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:37.256101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984e4d7-1f84-4269-84f6-76ab68adbd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:37.256707 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984e4d7-1f84-4269-84f6-76ab68adbd3d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:37.256998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984e4d7-1f84-4269-84f6-76ab68adbd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:37.257257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984e4d7-1f84-4269-84f6-76ab68adbd3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:37.259475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcd894-9e41-43ec-85d8-854a2f1436c7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:37.295008 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b300e41-6d31-4761-b838-714bdade1ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:37.295624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b300e41-6d31-4761-b838-714bdade1ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:37.295908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b300e41-6d31-4761-b838-714bdade1ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:37.296547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b300e41-6d31-4761-b838-714bdade1ae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:37.296973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b300e41-6d31-4761-b838-714bdade1ae8 None admin] RBAC: Adding query filter params (group.id=e58dbbfbc6ac4b338fc398aa98749792, role.id=854a7f7c663b4c66a08f7e731c7bf63c, scope.project.id=6409fa7c592e4bd09561cddf768267d4) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:37.297287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b300e41-6d31-4761-b838-714bdade1ae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:37.298719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b300e41-6d31-4761-b838-714bdade1ae8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:37.304690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 220/451] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jul 28 03:46:37 2022] GET /identity/v3/role_assignments?group.id=e58dbbfbc6ac4b338fc398aa98749792&scope.project.id=6409fa7c592e4bd09561cddf768267d4&role.id=854a7f7c663b4c66a08f7e731c7bf63c => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:37.305967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:37.325901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984e4d7-1f84-4269-84f6-76ab68adbd3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:37.327424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984e4d7-1f84-4269-84f6-76ab68adbd3d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:37.432318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 232/452] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 03:46:37.438171 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b300e41-6d31-4761-b838-714bdade1ae8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:37.541758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b300e41-6d31-4761-b838-714bdade1ae8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:37.568685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:37.577088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e077af5-b764-4929-9fc7-702983c68863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:37.577737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077af5-b764-4929-9fc7-702983c68863 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:37.578025 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077af5-b764-4929-9fc7-702983c68863 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:37.578309 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077af5-b764-4929-9fc7-702983c68863 None admin] PATH_INFO: `/v3/projects/826fca2fab6740f6870c6426848e8d90` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:37.579498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e077af5-b764-4929-9fc7-702983c68863 None admin] RBAC: Authorizing `identity:get_project(project_id=826fca2fab6740f6870c6426848e8d90)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:37.580937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e077af5-b764-4929-9fc7-702983c68863 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:37.582751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 221/453] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:46:37 2022] GET /identity/v3/projects/826fca2fab6740f6870c6426848e8d90 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:37.588948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984e4d7-1f84-4269-84f6-76ab68adbd3d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:37.671513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:37.689076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984e4d7-1f84-4269-84f6-76ab68adbd3d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:37.722051 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce363f1-7228-46c6-8992-00a10986fc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:37.722677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce363f1-7228-46c6-8992-00a10986fc3b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:37.722985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce363f1-7228-46c6-8992-00a10986fc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:37.723260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce363f1-7228-46c6-8992-00a10986fc3b None admin] PATH_INFO: `/v3/groups/3abac29c895647c699c4c15980e768c3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:37.724481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce363f1-7228-46c6-8992-00a10986fc3b None admin] RBAC: Authorizing `identity:get_group(group_id=3abac29c895647c699c4c15980e768c3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:37.725898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce363f1-7228-46c6-8992-00a10986fc3b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:37.727495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 233/454] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jul 28 03:46:37 2022] GET /identity/v3/groups/3abac29c895647c699c4c15980e768c3 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:37.733333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e077af5-b764-4929-9fc7-702983c68863 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:37.829851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e077af5-b764-4929-9fc7-702983c68863 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:37.861013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1d2e1-4a4e-48df-a0f8-811393e70fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:37.861617 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1d2e1-4a4e-48df-a0f8-811393e70fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:37.861898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1d2e1-4a4e-48df-a0f8-811393e70fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:37.862173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1d2e1-4a4e-48df-a0f8-811393e70fd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:37.862973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1d2e1-4a4e-48df-a0f8-811393e70fd7 None admin] RBAC: Adding query filter params (group.id=3abac29c895647c699c4c15980e768c3, role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=826fca2fab6740f6870c6426848e8d90) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:37.863285 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1d2e1-4a4e-48df-a0f8-811393e70fd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:37.865195 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1d2e1-4a4e-48df-a0f8-811393e70fd7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:37.870704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 222/455] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jul 28 03:46:37 2022] GET /identity/v3/role_assignments?group.id=3abac29c895647c699c4c15980e768c3&scope.project.id=826fca2fab6740f6870c6426848e8d90&role.id=a37ae24c516b48c69e13721c013134a2 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:40.226704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927d3e7-087f-412c-a7bc-b9a7599a2a30 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:40.227244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927d3e7-087f-412c-a7bc-b9a7599a2a30 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:40.227319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927d3e7-087f-412c-a7bc-b9a7599a2a30 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:40.228957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 234/456] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:40.234617 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4b52e-9573-49ed-8ee0-172a5933d407 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:40.235099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4b52e-9573-49ed-8ee0-172a5933d407 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:40.235290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4b52e-9573-49ed-8ee0-172a5933d407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:40.263768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b9c4b52e-9573-49ed-8ee0-172a5933d407 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:40.264969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b9c4b52e-9573-49ed-8ee0-172a5933d407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:40.341479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:40.359117 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 223/457] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:40.389147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b553a9-2a00-44ba-9576-b23e21b2f019 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:40.389501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b553a9-2a00-44ba-9576-b23e21b2f019 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:40.389856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b553a9-2a00-44ba-9576-b23e21b2f019 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:40.391470 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 235/458] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:40.393120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4b52e-9573-49ed-8ee0-172a5933d407 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:40.397854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691dfd1-65fb-4e80-a844-ed80cccf6701 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:40.398137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691dfd1-65fb-4e80-a844-ed80cccf6701 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:40.398415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691dfd1-65fb-4e80-a844-ed80cccf6701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:40.424278 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e691dfd1-65fb-4e80-a844-ed80cccf6701 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:40.425472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e691dfd1-65fb-4e80-a844-ed80cccf6701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:40.542869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 236/459] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:40.542869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eac773-af2a-4ce0-b9c8-7d2c4c925a9d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:40.542869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eac773-af2a-4ce0-b9c8-7d2c4c925a9d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:40.543135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eac773-af2a-4ce0-b9c8-7d2c4c925a9d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:40.544839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 237/460] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:40.562809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd65f23-7f12-4cd9-8314-ff70b819e750 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:40.562809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd65f23-7f12-4cd9-8314-ff70b819e750 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:40.562809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd65f23-7f12-4cd9-8314-ff70b819e750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:40.568403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4b52e-9573-49ed-8ee0-172a5933d407 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:40.581754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fcd65f23-7f12-4cd9-8314-ff70b819e750 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:40.583220 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fcd65f23-7f12-4cd9-8314-ff70b819e750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:40.604165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03227a7f-3648-472b-83c1-1e410391f315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:40.604922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03227a7f-3648-472b-83c1-1e410391f315 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:40.605211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03227a7f-3648-472b-83c1-1e410391f315 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:40.605488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03227a7f-3648-472b-83c1-1e410391f315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:40.617032 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03227a7f-3648-472b-83c1-1e410391f315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:40.618512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03227a7f-3648-472b-83c1-1e410391f315 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-03227a7f-3648-472b-83c1-1e410391f315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:40.636061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:40.636987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:40.637844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:40.637844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:40.637844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:40.637844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:40.637844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 224/461] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:40.642328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03227a7f-3648-472b-83c1-1e410391f315 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:40.654865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 238/462] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:40.655597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd65f23-7f12-4cd9-8314-ff70b819e750 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:40.667419 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:40.715738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:40.758545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd65f23-7f12-4cd9-8314-ff70b819e750 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:40.780385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03227a7f-3648-472b-83c1-1e410391f315 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:40.794731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6e828-2ee6-42fa-8e09-57a267e321f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:40.795383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6e828-2ee6-42fa-8e09-57a267e321f4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:40.795664 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6e828-2ee6-42fa-8e09-57a267e321f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:40.795940 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6e828-2ee6-42fa-8e09-57a267e321f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:40.796642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6e828-2ee6-42fa-8e09-57a267e321f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:40.796950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6e828-2ee6-42fa-8e09-57a267e321f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:40.798338 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6e828-2ee6-42fa-8e09-57a267e321f4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:40.810392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 239/463] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:40.811070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6e828-2ee6-42fa-8e09-57a267e321f4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:40.820930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f65f4-e075-4daa-9bec-1376248898ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:40.821585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f65f4-e075-4daa-9bec-1376248898ad None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:40.821864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f65f4-e075-4daa-9bec-1376248898ad None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:40.822135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f65f4-e075-4daa-9bec-1376248898ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:40.834482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f65f4-e075-4daa-9bec-1376248898ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:40.836044 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f65f4-e075-4daa-9bec-1376248898ad None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5d3f65f4-e075-4daa-9bec-1376248898ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:40.860625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:40.862254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:40.863957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:40.863957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:40.863957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:40.863957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:40.870808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 225/464] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:40.870808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f65f4-e075-4daa-9bec-1376248898ad None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:40.895341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:40.936505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6e828-2ee6-42fa-8e09-57a267e321f4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:40.984414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610274fd-8818-450f-9ebe-58d4ff3043a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:40.985065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610274fd-8818-450f-9ebe-58d4ff3043a1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:40.985345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610274fd-8818-450f-9ebe-58d4ff3043a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:40.985619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610274fd-8818-450f-9ebe-58d4ff3043a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:40.993103 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:40.999595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610274fd-8818-450f-9ebe-58d4ff3043a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:41.000889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f65f4-e075-4daa-9bec-1376248898ad None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:41.001107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610274fd-8818-450f-9ebe-58d4ff3043a1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-610274fd-8818-450f-9ebe-58d4ff3043a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:41.018996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:41.019923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:41.020796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:41.020796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:41.020796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:41.020796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:41.020796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 240/465] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:41.021829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610274fd-8818-450f-9ebe-58d4ff3043a1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:41.044579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb78db-f2dc-4dad-a099-c1952899238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:41.046246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb78db-f2dc-4dad-a099-c1952899238e None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:41.046737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb78db-f2dc-4dad-a099-c1952899238e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:41.047313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb78db-f2dc-4dad-a099-c1952899238e None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:41.048345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb78db-f2dc-4dad-a099-c1952899238e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:41.050126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb78db-f2dc-4dad-a099-c1952899238e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:41.086225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 226/466] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:46:40 2022] POST /identity/v3/users => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:46:41.087476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb78db-f2dc-4dad-a099-c1952899238e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:41.145837 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610274fd-8818-450f-9ebe-58d4ff3043a1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:41.172525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:41.186180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728a39d-dc12-4c59-a422-68cdbea1ee81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:41.186834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728a39d-dc12-4c59-a422-68cdbea1ee81 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:41.187378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728a39d-dc12-4c59-a422-68cdbea1ee81 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:41.187378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728a39d-dc12-4c59-a422-68cdbea1ee81 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:41.188107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728a39d-dc12-4c59-a422-68cdbea1ee81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:41.188417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728a39d-dc12-4c59-a422-68cdbea1ee81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:41.189830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728a39d-dc12-4c59-a422-68cdbea1ee81 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:41.203568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 241/467] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:41.204794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634b05-370c-4ffc-98aa-6fe5ac735d88 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:41.205090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634b05-370c-4ffc-98aa-6fe5ac735d88 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:41.205371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634b05-370c-4ffc-98aa-6fe5ac735d88 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:41.211547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 242/468] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:41.212211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec634b05-370c-4ffc-98aa-6fe5ac735d88 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:41.222435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb78db-f2dc-4dad-a099-c1952899238e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:41.285748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f8aa0-be56-458e-9852-dbd54d9e237b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:41.287775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f8aa0-be56-458e-9852-dbd54d9e237b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:41.287775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f8aa0-be56-458e-9852-dbd54d9e237b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:41.287775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f8aa0-be56-458e-9852-dbd54d9e237b None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:41.288465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f8aa0-be56-458e-9852-dbd54d9e237b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:41.288775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f8aa0-be56-458e-9852-dbd54d9e237b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:41.290168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f8aa0-be56-458e-9852-dbd54d9e237b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:41.310966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 227/469] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:41.312333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e2381-0246-4d80-9d74-773807227998 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:41.312627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e2381-0246-4d80-9d74-773807227998 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:41.312908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e2381-0246-4d80-9d74-773807227998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:41.337089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ea6e2381-0246-4d80-9d74-773807227998 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:41.338293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ea6e2381-0246-4d80-9d74-773807227998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:41.347259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:41.348350 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec634b05-370c-4ffc-98aa-6fe5ac735d88 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:41.394742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594222e-e57c-45f9-b2b2-bd6fade63545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:41.396394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594222e-e57c-45f9-b2b2-bd6fade63545 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:41.396931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594222e-e57c-45f9-b2b2-bd6fade63545 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:41.397445 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594222e-e57c-45f9-b2b2-bd6fade63545 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:41.398328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594222e-e57c-45f9-b2b2-bd6fade63545 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:41.400112 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594222e-e57c-45f9-b2b2-bd6fade63545 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:41.420558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 228/470] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:41.421455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e2381-0246-4d80-9d74-773807227998 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:41.431330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 243/471] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:46:41 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:46:41.459587 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594222e-e57c-45f9-b2b2-bd6fade63545 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:41.515635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:41.600153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594222e-e57c-45f9-b2b2-bd6fade63545 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:41.639004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa409cad-3270-47a7-b26a-7cb0064e8a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:41.639785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e2381-0246-4d80-9d74-773807227998 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:41.640227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa409cad-3270-47a7-b26a-7cb0064e8a0d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:41.640797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa409cad-3270-47a7-b26a-7cb0064e8a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:41.641377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa409cad-3270-47a7-b26a-7cb0064e8a0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:41.655730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa409cad-3270-47a7-b26a-7cb0064e8a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:41.657764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa409cad-3270-47a7-b26a-7cb0064e8a0d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-fa409cad-3270-47a7-b26a-7cb0064e8a0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:41.676412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:41.678102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:41.679010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:41.679010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:41.679010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:41.679010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:41.681806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 244/472] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:41.686930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa409cad-3270-47a7-b26a-7cb0064e8a0d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:41.689626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ef199-8477-4b09-bda6-37ff63892a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:41.690263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ef199-8477-4b09-bda6-37ff63892a77 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:41.690558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ef199-8477-4b09-bda6-37ff63892a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:41.690844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ef199-8477-4b09-bda6-37ff63892a77 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:41.691786 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ef199-8477-4b09-bda6-37ff63892a77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:41.698921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ef199-8477-4b09-bda6-37ff63892a77 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:41.715143 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:41.726621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 229/473] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:46:41 2022] POST /identity/v3/users => generated 248 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:46:41.813157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:41.818120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa409cad-3270-47a7-b26a-7cb0064e8a0d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:41.858126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2309c7-8653-4ba7-b0e0-5854f9e7882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:41.858678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2309c7-8653-4ba7-b0e0-5854f9e7882f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:41.859000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2309c7-8653-4ba7-b0e0-5854f9e7882f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:41.859277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2309c7-8653-4ba7-b0e0-5854f9e7882f None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:41.859995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2309c7-8653-4ba7-b0e0-5854f9e7882f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:41.860302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2309c7-8653-4ba7-b0e0-5854f9e7882f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:41.861846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2309c7-8653-4ba7-b0e0-5854f9e7882f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:41.876044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 245/474] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:41.881956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ef199-8477-4b09-bda6-37ff63892a77 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:41.997290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ef199-8477-4b09-bda6-37ff63892a77 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:42.034242 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547bdbd-e885-4632-ab18-4636d50415c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:42.034918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547bdbd-e885-4632-ab18-4636d50415c0 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:42.035203 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547bdbd-e885-4632-ab18-4636d50415c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:42.035483 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547bdbd-e885-4632-ab18-4636d50415c0 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:42.037099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547bdbd-e885-4632-ab18-4636d50415c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:42.038557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547bdbd-e885-4632-ab18-4636d50415c0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:42.061960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd39d-0051-493a-9f05-f4a83fbb78f5 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:42.062309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd39d-0051-493a-9f05-f4a83fbb78f5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:42.062592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd39d-0051-493a-9f05-f4a83fbb78f5 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:42.064328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 246/475] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:46:42.064627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 230/476] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:46:41 2022] POST /identity/v3/users => generated 249 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:46:42.070471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefaab76-93a2-4424-8284-f8f6e4b718dc None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:42.070768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefaab76-93a2-4424-8284-f8f6e4b718dc None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:42.071127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefaab76-93a2-4424-8284-f8f6e4b718dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:42.091772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fefaab76-93a2-4424-8284-f8f6e4b718dc None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:42.093027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fefaab76-93a2-4424-8284-f8f6e4b718dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:42.174674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 247/477] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:42.200683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547bdbd-e885-4632-ab18-4636d50415c0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:42.263621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:42.316284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547bdbd-e885-4632-ab18-4636d50415c0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:42.352435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055af97-b4c7-4778-8c23-d1d555bb6e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:42.353124 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055af97-b4c7-4778-8c23-d1d555bb6e19 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:42.353408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055af97-b4c7-4778-8c23-d1d555bb6e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:42.353683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055af97-b4c7-4778-8c23-d1d555bb6e19 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:42.354393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055af97-b4c7-4778-8c23-d1d555bb6e19 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:42.355858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055af97-b4c7-4778-8c23-d1d555bb6e19 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:42.363712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.sql.core [None req-d055af97-b4c7-4778-8c23-d1d555bb6e19 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 03:46:42.363712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 03:46:42.363712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [parameters: {'id': '88944dc75dc342c8824e6dbc59fcf0cf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 03:46:42.363712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80472) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d055af97-b4c7-4778-8c23-d1d555bb6e19 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 28 03:46:42.381240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 03:46:42.382144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application yield resource Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise self.value Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application session.commit() Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 03:46:42.383100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.flush() Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application compat.raise_( Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 03:46:42.384521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 03:46:42.385680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 03:46:42.386527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [parameters: {'id': '88944dc75dc342c8824e6dbc59fcf0cf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:42.387367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 03:46:42.391769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:42.392742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 231/478] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Jul 28 03:46:42 2022] POST /identity/v3/roles => generated 144 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:42.392890 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefaab76-93a2-4424-8284-f8f6e4b718dc None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:42.483864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:42.505694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefaab76-93a2-4424-8284-f8f6e4b718dc None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:42.546814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad90054-c54a-4518-be48-d55b6fd67fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:42.547472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad90054-c54a-4518-be48-d55b6fd67fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:42.547751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad90054-c54a-4518-be48-d55b6fd67fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:42.548022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad90054-c54a-4518-be48-d55b6fd67fb8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:42.559505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad90054-c54a-4518-be48-d55b6fd67fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-5ad90054-c54a-4518-be48-d55b6fd67fb8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:42.565387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:42.566258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 03:46:42.567113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:42.569150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 248/479] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:46:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:42.573595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055af97-b4c7-4778-8c23-d1d555bb6e19 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:42.710876 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055af97-b4c7-4778-8c23-d1d555bb6e19 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:42.727892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:42.754607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51bead-5246-4ad0-9df1-e829e3339050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:42.755330 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51bead-5246-4ad0-9df1-e829e3339050 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:42.755787 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51bead-5246-4ad0-9df1-e829e3339050 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:42.757353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51bead-5246-4ad0-9df1-e829e3339050 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:42.758083 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51bead-5246-4ad0-9df1-e829e3339050 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:42.758437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51bead-5246-4ad0-9df1-e829e3339050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:42.759834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51bead-5246-4ad0-9df1-e829e3339050 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:42.768575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 232/480] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:43.618914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498e82f-6cb6-4c47-9580-360ff3693ef9 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:43.619416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498e82f-6cb6-4c47-9580-360ff3693ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:43.619517 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498e82f-6cb6-4c47-9580-360ff3693ef9 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:43.621143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 249/481] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:43.628673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fab6f-379b-43c2-aac6-70bc3fb107cc None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:43.629086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fab6f-379b-43c2-aac6-70bc3fb107cc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:43.629259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fab6f-379b-43c2-aac6-70bc3fb107cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:43.655841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-019fab6f-379b-43c2-aac6-70bc3fb107cc None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:43.657051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-019fab6f-379b-43c2-aac6-70bc3fb107cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:43.741241 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 233/482] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:43.788389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498e82f-6cb6-4c47-9580-360ff3693ef9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:43.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498e82f-6cb6-4c47-9580-360ff3693ef9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:43.964073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51efe4-4903-4303-92e5-63fb77e8f8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:43.964716 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51efe4-4903-4303-92e5-63fb77e8f8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:43.964997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51efe4-4903-4303-92e5-63fb77e8f8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:43.965270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51efe4-4903-4303-92e5-63fb77e8f8c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:43.973086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:43.983135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51efe4-4903-4303-92e5-63fb77e8f8c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3b51efe4-4903-4303-92e5-63fb77e8f8c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:43.988356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:43.989293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:43.990471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:43.993491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 250/483] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:43.999680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fab6f-379b-43c2-aac6-70bc3fb107cc None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:44.104030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fab6f-379b-43c2-aac6-70bc3fb107cc None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:44.105246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb506f8a-bb8a-46fa-aead-8f5598141cc7 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.105759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb506f8a-bb8a-46fa-aead-8f5598141cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.106244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb506f8a-bb8a-46fa-aead-8f5598141cc7 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.109082 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 251/484] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:46:44.113538 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffae3e4-0638-434d-85f7-6268cdee3fc0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.114359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffae3e4-0638-434d-85f7-6268cdee3fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.115150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffae3e4-0638-434d-85f7-6268cdee3fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.135152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:44.139348 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007efc46-3de7-4904-9810-2248a1ee2485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:44.140023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007efc46-3de7-4904-9810-2248a1ee2485 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.142880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007efc46-3de7-4904-9810-2248a1ee2485 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.142880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007efc46-3de7-4904-9810-2248a1ee2485 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.142880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007efc46-3de7-4904-9810-2248a1ee2485 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:44.142880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007efc46-3de7-4904-9810-2248a1ee2485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:44.143141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007efc46-3de7-4904-9810-2248a1ee2485 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:44.152724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 234/485] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:44.158443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007efc46-3de7-4904-9810-2248a1ee2485 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:44.166824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cffae3e4-0638-434d-85f7-6268cdee3fc0 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:44.168043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cffae3e4-0638-434d-85f7-6268cdee3fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:44.286915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 252/486] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:44.307734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffae3e4-0638-434d-85f7-6268cdee3fc0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:44.324962 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007efc46-3de7-4904-9810-2248a1ee2485 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:44.398464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef0360-a15b-4a33-b23c-ba0a449083ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:44.399174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef0360-a15b-4a33-b23c-ba0a449083ab None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.399548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef0360-a15b-4a33-b23c-ba0a449083ab None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.399827 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef0360-a15b-4a33-b23c-ba0a449083ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.412407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef0360-a15b-4a33-b23c-ba0a449083ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:44.414045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef0360-a15b-4a33-b23c-ba0a449083ab None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-faef0360-a15b-4a33-b23c-ba0a449083ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:44.430030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:44.430947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:44.432200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:44.432200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:44.432200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:44.432200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:44.432415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffae3e4-0638-434d-85f7-6268cdee3fc0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:44.432486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 235/487] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:44.433652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c699e-814e-4d91-bedc-1e13eb2cd35c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.433652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c699e-814e-4d91-bedc-1e13eb2cd35c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.433652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c699e-814e-4d91-bedc-1e13eb2cd35c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.438757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 236/488] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:44.439669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c699e-814e-4d91-bedc-1e13eb2cd35c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:44.461105 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:44.468454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048b89c-8f8e-40e2-a67c-72eaa1e33e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:44.468946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:44.469094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048b89c-8f8e-40e2-a67c-72eaa1e33e3c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.469380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048b89c-8f8e-40e2-a67c-72eaa1e33e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.469659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048b89c-8f8e-40e2-a67c-72eaa1e33e3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.482433 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048b89c-8f8e-40e2-a67c-72eaa1e33e3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-0048b89c-8f8e-40e2-a67c-72eaa1e33e3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:44.487546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:44.488426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:44.489523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 253/489] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:44.490655 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a2de-d98b-4807-84ba-91f3279fa462 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.490655 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a2de-d98b-4807-84ba-91f3279fa462 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.490916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a2de-d98b-4807-84ba-91f3279fa462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.510299 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-46b5a2de-d98b-4807-84ba-91f3279fa462 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:44.511637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-46b5a2de-d98b-4807-84ba-91f3279fa462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:44.559496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c699e-814e-4d91-bedc-1e13eb2cd35c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:44.585759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 254/490] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:44.586674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5a2de-d98b-4807-84ba-91f3279fa462 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:44.615850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb39b-9236-439a-9fd9-00b2a92b2a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:44.616716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb39b-9236-439a-9fd9-00b2a92b2a6d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.617206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb39b-9236-439a-9fd9-00b2a92b2a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.617669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb39b-9236-439a-9fd9-00b2a92b2a6d None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.618467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8eb39b-9236-439a-9fd9-00b2a92b2a6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:44.618894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8eb39b-9236-439a-9fd9-00b2a92b2a6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:44.624522 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8eb39b-9236-439a-9fd9-00b2a92b2a6d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:44.639060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 237/491] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:44.640191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb39b-9236-439a-9fd9-00b2a92b2a6d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:44.696469 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5a2de-d98b-4807-84ba-91f3279fa462 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:44.718317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:44.728955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:44.737120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6338c8b-3723-4be8-9f54-14b9514acf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:44.737746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6338c8b-3723-4be8-9f54-14b9514acf20 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.738028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6338c8b-3723-4be8-9f54-14b9514acf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.738302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6338c8b-3723-4be8-9f54-14b9514acf20 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.739609 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6338c8b-3723-4be8-9f54-14b9514acf20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:44.739921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6338c8b-3723-4be8-9f54-14b9514acf20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:44.741539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6338c8b-3723-4be8-9f54-14b9514acf20 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:44.750129 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb39b-9236-439a-9fd9-00b2a92b2a6d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:44.750341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 255/492] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:44.751187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6338c8b-3723-4be8-9f54-14b9514acf20 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:44.787914 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c4230-e9ec-41c8-8169-22dc3803d1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:44.788647 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c4230-e9ec-41c8-8169-22dc3803d1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.788931 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c4230-e9ec-41c8-8169-22dc3803d1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.789307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c4230-e9ec-41c8-8169-22dc3803d1b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.800325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c4230-e9ec-41c8-8169-22dc3803d1b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f23c4230-e9ec-41c8-8169-22dc3803d1b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:44.806351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:44.808431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:44.810380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:44.818809 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 238/493] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:44.818809 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c4230-e9ec-41c8-8169-22dc3803d1b7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:44.857790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6338c8b-3723-4be8-9f54-14b9514acf20 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:44.893114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fd820-29fa-4cb0-84b6-8c4f9277f3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:44.893811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fd820-29fa-4cb0-84b6-8c4f9277f3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.894098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fd820-29fa-4cb0-84b6-8c4f9277f3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.894374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fd820-29fa-4cb0-84b6-8c4f9277f3d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.902647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fd820-29fa-4cb0-84b6-8c4f9277f3d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:44.904144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fd820-29fa-4cb0-84b6-8c4f9277f3d6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-321fd820-29fa-4cb0-84b6-8c4f9277f3d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:44.917931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:44.919092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:44.920024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:44.920024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:44.920024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:44.920024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:44.920024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:44.920024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:44.920024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:44.920024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:44.920024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:46:44.920024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:44.920024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 256/494] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jul 28 03:46:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:44.920566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fd820-29fa-4cb0-84b6-8c4f9277f3d6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:44.936502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c4230-e9ec-41c8-8169-22dc3803d1b7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:44.968748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:44.975177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ca49a-b0ed-4713-8903-3c047a64d9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:44.975987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ca49a-b0ed-4713-8903-3c047a64d9dc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:44.976408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ca49a-b0ed-4713-8903-3c047a64d9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:44.976808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ca49a-b0ed-4713-8903-3c047a64d9dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:44.987341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ca49a-b0ed-4713-8903-3c047a64d9dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:44.988945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ca49a-b0ed-4713-8903-3c047a64d9dc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:44.991500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3d9ca49a-b0ed-4713-8903-3c047a64d9dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:45.007861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:45.008753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:45.009572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:45.009572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:45.009572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:45.009572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:45.009572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 239/495] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:45.010624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ccd15-4943-4829-a6e2-5bf533c8023c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.010939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ccd15-4943-4829-a6e2-5bf533c8023c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.011217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ccd15-4943-4829-a6e2-5bf533c8023c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.012615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 240/496] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:45.013152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ccd15-4943-4829-a6e2-5bf533c8023c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:45.037692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fd820-29fa-4cb0-84b6-8c4f9277f3d6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:45.072057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e6fb8-661a-47c1-b58a-0822bc9d0671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:45.072678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e6fb8-661a-47c1-b58a-0822bc9d0671 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.072979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e6fb8-661a-47c1-b58a-0822bc9d0671 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.073239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e6fb8-661a-47c1-b58a-0822bc9d0671 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.073958 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e6fb8-661a-47c1-b58a-0822bc9d0671 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:45.074383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e6fb8-661a-47c1-b58a-0822bc9d0671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:45.075801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e6fb8-661a-47c1-b58a-0822bc9d0671 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:45.086820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 257/497] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:45.086820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e6fb8-661a-47c1-b58a-0822bc9d0671 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:45.134555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ccd15-4943-4829-a6e2-5bf533c8023c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:45.172029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f988db5-55b1-4f9f-8653-d8cd8838c41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:45.172759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f988db5-55b1-4f9f-8653-d8cd8838c41a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.173041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f988db5-55b1-4f9f-8653-d8cd8838c41a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.173317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f988db5-55b1-4f9f-8653-d8cd8838c41a None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.174110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f988db5-55b1-4f9f-8653-d8cd8838c41a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:45.174417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f988db5-55b1-4f9f-8653-d8cd8838c41a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:45.175999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f988db5-55b1-4f9f-8653-d8cd8838c41a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:45.187028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e6fb8-661a-47c1-b58a-0822bc9d0671 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:45.189356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:45.220922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 241/498] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jul 28 03:46:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:45.220922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96632-034e-49aa-a1fb-d56fdea057bd None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.220922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96632-034e-49aa-a1fb-d56fdea057bd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.220922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96632-034e-49aa-a1fb-d56fdea057bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.231662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4017a0-bf3c-4f51-8c26-8fa0eef59e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:45.232692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4017a0-bf3c-4f51-8c26-8fa0eef59e8e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.232979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4017a0-bf3c-4f51-8c26-8fa0eef59e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.233260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4017a0-bf3c-4f51-8c26-8fa0eef59e8e None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.233964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4017a0-bf3c-4f51-8c26-8fa0eef59e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:45.234271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4017a0-bf3c-4f51-8c26-8fa0eef59e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:45.235684 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4017a0-bf3c-4f51-8c26-8fa0eef59e8e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:45.238027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8dd96632-034e-49aa-a1fb-d56fdea057bd None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:45.239723 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8dd96632-034e-49aa-a1fb-d56fdea057bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:45.252545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 258/499] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:45.252545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4017a0-bf3c-4f51-8c26-8fa0eef59e8e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:45.297090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:45.319020 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 242/500] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:45.320086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd96632-034e-49aa-a1fb-d56fdea057bd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:45.375460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4017a0-bf3c-4f51-8c26-8fa0eef59e8e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:45.414491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0e56f-643e-41ef-8fcb-17e19715e322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:45.415163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0e56f-643e-41ef-8fcb-17e19715e322 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.415449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0e56f-643e-41ef-8fcb-17e19715e322 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.415956 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0e56f-643e-41ef-8fcb-17e19715e322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.425929 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0e56f-643e-41ef-8fcb-17e19715e322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:45.427689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0e56f-643e-41ef-8fcb-17e19715e322 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:45.429324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd96632-034e-49aa-a1fb-d56fdea057bd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-f2e0e56f-643e-41ef-8fcb-17e19715e322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:45.443139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:45.444163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:45.445126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:45.445126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:45.445126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:45.445126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:45.446613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 259/501] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:45.447641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0e56f-643e-41ef-8fcb-17e19715e322 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:45.470280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:45.473208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebef1d-c375-4ec0-a92d-0c12fca60a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:45.474964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebef1d-c375-4ec0-a92d-0c12fca60a8d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.475263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebef1d-c375-4ec0-a92d-0c12fca60a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.475542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebef1d-c375-4ec0-a92d-0c12fca60a8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.528821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:45.553623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebef1d-c375-4ec0-a92d-0c12fca60a8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:45.555126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebef1d-c375-4ec0-a92d-0c12fca60a8d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:45.558482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0e56f-643e-41ef-8fcb-17e19715e322 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:45.592254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7cfdf-91e3-4b83-94ce-6f5d55a20059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:45.592893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7cfdf-91e3-4b83-94ce-6f5d55a20059 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.593177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7cfdf-91e3-4b83-94ce-6f5d55a20059 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.593454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7cfdf-91e3-4b83-94ce-6f5d55a20059 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.601928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7cfdf-91e3-4b83-94ce-6f5d55a20059 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:45.603409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7cfdf-91e3-4b83-94ce-6f5d55a20059 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-73d7cfdf-91e3-4b83-94ce-6f5d55a20059 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:45.614808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:45.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:45.616595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:45.616595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:45.616595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:45.616595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:45.616595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:45.616595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:45.616595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:45.616595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:45.616595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 03:46:45.616595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:45.616595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 260/502] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:45.617149 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7cfdf-91e3-4b83-94ce-6f5d55a20059 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:45.658443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 243/503] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:45.659323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebef1d-c375-4ec0-a92d-0c12fca60a8d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:45.694079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:45.717497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7cfdf-91e3-4b83-94ce-6f5d55a20059 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:45.741407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:45.750046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a2cbc-e500-4002-a6b6-09ce9dfc3478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:45.750679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a2cbc-e500-4002-a6b6-09ce9dfc3478 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.750983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a2cbc-e500-4002-a6b6-09ce9dfc3478 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.751264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a2cbc-e500-4002-a6b6-09ce9dfc3478 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.760212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebef1d-c375-4ec0-a92d-0c12fca60a8d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:45.761490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a2cbc-e500-4002-a6b6-09ce9dfc3478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-0c5a2cbc-e500-4002-a6b6-09ce9dfc3478 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:45.767527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:45.768629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:45.769582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 261/504] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:45.770978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489a224-4da8-4831-9881-a9573b4f1dd6 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.770978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489a224-4da8-4831-9881-a9573b4f1dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.771148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489a224-4da8-4831-9881-a9573b4f1dd6 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.772450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 262/505] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:45.773006 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489a224-4da8-4831-9881-a9573b4f1dd6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:45.801252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c35cf9-65d0-4396-8b66-a0d4b1cd8511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:45.802211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c35cf9-65d0-4396-8b66-a0d4b1cd8511 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.802499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c35cf9-65d0-4396-8b66-a0d4b1cd8511 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.802775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c35cf9-65d0-4396-8b66-a0d4b1cd8511 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.803491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c35cf9-65d0-4396-8b66-a0d4b1cd8511 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:45.803803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c35cf9-65d0-4396-8b66-a0d4b1cd8511 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:45.805180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c35cf9-65d0-4396-8b66-a0d4b1cd8511 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:45.821609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 244/506] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:45.821609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c35cf9-65d0-4396-8b66-a0d4b1cd8511 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:45.875514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489a224-4da8-4831-9881-a9573b4f1dd6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:45.908754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa13cbc-e11a-4954-9c74-837237990be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:45.909375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13cbc-e11a-4954-9c74-837237990be5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.909660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13cbc-e11a-4954-9c74-837237990be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.909932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13cbc-e11a-4954-9c74-837237990be5 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.910729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa13cbc-e11a-4954-9c74-837237990be5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:45.911823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa13cbc-e11a-4954-9c74-837237990be5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:45.913270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa13cbc-e11a-4954-9c74-837237990be5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:45.915601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c35cf9-65d0-4396-8b66-a0d4b1cd8511 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:45.924769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:45.949902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a7f7c-8d89-4c8d-b26b-71d8119dec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:45.950445 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 263/507] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jul 28 03:46:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:45.950587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a7f7c-8d89-4c8d-b26b-71d8119dec6e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.950850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a7f7c-8d89-4c8d-b26b-71d8119dec6e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.951135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a7f7c-8d89-4c8d-b26b-71d8119dec6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:45.951290 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa13cbc-e11a-4954-9c74-837237990be5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:45.964745 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a7f7c-8d89-4c8d-b26b-71d8119dec6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:45.966208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a7f7c-8d89-4c8d-b26b-71d8119dec6e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:45.967945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-4f4a7f7c-8d89-4c8d-b26b-71d8119dec6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:45.981625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:45.982466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:45.983395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:45.983395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:45.983395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:45.983395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:45.983395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 245/508] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:45.984360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5970536-f593-48a7-ad85-529f48d95e0b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:45.984360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5970536-f593-48a7-ad85-529f48d95e0b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:45.984596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5970536-f593-48a7-ad85-529f48d95e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:46.002908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a5970536-f593-48a7-ad85-529f48d95e0b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:46.004040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a5970536-f593-48a7-ad85-529f48d95e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:46.054423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa13cbc-e11a-4954-9c74-837237990be5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:46.070419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 246/509] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:46.090984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5970536-f593-48a7-ad85-529f48d95e0b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:46.091067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89db4a-2144-4554-8fe3-8aa4f1372f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:46.091067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89db4a-2144-4554-8fe3-8aa4f1372f07 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:46.091067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89db4a-2144-4554-8fe3-8aa4f1372f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:46.091067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89db4a-2144-4554-8fe3-8aa4f1372f07 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:46.091350 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89db4a-2144-4554-8fe3-8aa4f1372f07 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:46.091350 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89db4a-2144-4554-8fe3-8aa4f1372f07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:46.092781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89db4a-2144-4554-8fe3-8aa4f1372f07 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:46.101556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 264/510] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:46.102375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89db4a-2144-4554-8fe3-8aa4f1372f07 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:46.159805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:46.170852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5970536-f593-48a7-ad85-529f48d95e0b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:46.193337 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:46.204053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa2bf5-a138-4ed7-a89f-62fbb6ba5696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:46.204232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89db4a-2144-4554-8fe3-8aa4f1372f07 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:46.204680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa2bf5-a138-4ed7-a89f-62fbb6ba5696 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:46.204959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa2bf5-a138-4ed7-a89f-62fbb6ba5696 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:46.205229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa2bf5-a138-4ed7-a89f-62fbb6ba5696 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:46.212919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa2bf5-a138-4ed7-a89f-62fbb6ba5696 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:46.214470 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa2bf5-a138-4ed7-a89f-62fbb6ba5696 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-38fa2bf5-a138-4ed7-a89f-62fbb6ba5696 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:46.228022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:46.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:46.229840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:46.229840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:46.229840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:46.229840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:46.229840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:46.229840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:46.229840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:46.229840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:46.229840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 03:46:46.229840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:46.229840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 247/511] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:46.230397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa2bf5-a138-4ed7-a89f-62fbb6ba5696 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:46.244191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46817eda-32f5-43e5-9156-90c4cd401706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:46.244811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46817eda-32f5-43e5-9156-90c4cd401706 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:46.245095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46817eda-32f5-43e5-9156-90c4cd401706 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:46.245372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46817eda-32f5-43e5-9156-90c4cd401706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:46.313067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46817eda-32f5-43e5-9156-90c4cd401706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:46.314548 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46817eda-32f5-43e5-9156-90c4cd401706 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:46.329728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa2bf5-a138-4ed7-a89f-62fbb6ba5696 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:46.363527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4a0e8-6aa3-4687-930e-ae624b9bf969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:46.364960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4a0e8-6aa3-4687-930e-ae624b9bf969 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:46.365409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4a0e8-6aa3-4687-930e-ae624b9bf969 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:46.365698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4a0e8-6aa3-4687-930e-ae624b9bf969 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:46.366385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4a0e8-6aa3-4687-930e-ae624b9bf969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:46.366693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4a0e8-6aa3-4687-930e-ae624b9bf969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:46.367843 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:46.368113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4a0e8-6aa3-4687-930e-ae624b9bf969 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:46.380676 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 248/512] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:46.381602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4a0e8-6aa3-4687-930e-ae624b9bf969 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:46.415159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 265/513] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:46.415997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46817eda-32f5-43e5-9156-90c4cd401706 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:46.432912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:46.487599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4a0e8-6aa3-4687-930e-ae624b9bf969 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:46.514237 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46817eda-32f5-43e5-9156-90c4cd401706 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:46.522411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d536f-a9f1-43e3-9763-002c0b413ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:46.523074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d536f-a9f1-43e3-9763-002c0b413ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:46.523714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d536f-a9f1-43e3-9763-002c0b413ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:46.524000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d536f-a9f1-43e3-9763-002c0b413ab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:46.536985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d536f-a9f1-43e3-9763-002c0b413ab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:46.538468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d536f-a9f1-43e3-9763-002c0b413ab9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:46.548286 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93900e0a-bc9e-4fc2-9ebd-ad203ed20ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:46.548920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93900e0a-bc9e-4fc2-9ebd-ad203ed20ffe None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:46.549228 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93900e0a-bc9e-4fc2-9ebd-ad203ed20ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:46.549499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93900e0a-bc9e-4fc2-9ebd-ad203ed20ffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ae2d536f-a9f1-43e3-9763-002c0b413ab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:46.553749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:46.554558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:46.555435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:46.555435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:46.555435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:46.555435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:46.555435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 249/514] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:46.556095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d536f-a9f1-43e3-9763-002c0b413ab9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:46.563659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93900e0a-bc9e-4fc2-9ebd-ad203ed20ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:46.565061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93900e0a-bc9e-4fc2-9ebd-ad203ed20ffe None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-93900e0a-bc9e-4fc2-9ebd-ad203ed20ffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:46.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:46.581071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:46.582093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:46.582093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:46.582093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:46.582093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:46.582093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 266/515] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:46.582523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93900e0a-bc9e-4fc2-9ebd-ad203ed20ffe None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:46.596253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:46.652388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:46.658333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d536f-a9f1-43e3-9763-002c0b413ab9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:46.687661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93900e0a-bc9e-4fc2-9ebd-ad203ed20ffe None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:46.691670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7e6da-479b-4c86-97c1-202210cba055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:46.692326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7e6da-479b-4c86-97c1-202210cba055 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:46.692607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7e6da-479b-4c86-97c1-202210cba055 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:46.692871 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7e6da-479b-4c86-97c1-202210cba055 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:46.693651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7e6da-479b-4c86-97c1-202210cba055 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:46.693957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7e6da-479b-4c86-97c1-202210cba055 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:46.695416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7e6da-479b-4c86-97c1-202210cba055 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:46.712244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 250/516] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jul 28 03:46:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:46.713026 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7e6da-479b-4c86-97c1-202210cba055 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:46.725893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23f68a-9294-4d6a-80f7-b0c981edd15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:46.726653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23f68a-9294-4d6a-80f7-b0c981edd15d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:46.726834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23f68a-9294-4d6a-80f7-b0c981edd15d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:46.727081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23f68a-9294-4d6a-80f7-b0c981edd15d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:46.736253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23f68a-9294-4d6a-80f7-b0c981edd15d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:46.738613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23f68a-9294-4d6a-80f7-b0c981edd15d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1c23f68a-9294-4d6a-80f7-b0c981edd15d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:46:46.753079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:46.753921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:46.754845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:46:46.754845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:46:46.754845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:46:46.754845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:46:46.754845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:46:46.754845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:46:46.754845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:46.754845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:46.755268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 267/517] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:46.755814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23f68a-9294-4d6a-80f7-b0c981edd15d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:46.815480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7e6da-479b-4c86-97c1-202210cba055 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:46.821476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:46.850220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef0739-2b8b-47ec-ac5b-9116e121199b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:46.851155 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0739-2b8b-47ec-ac5b-9116e121199b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:46.851438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0739-2b8b-47ec-ac5b-9116e121199b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:46.851716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0739-2b8b-47ec-ac5b-9116e121199b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:46.856484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23f68a-9294-4d6a-80f7-b0c981edd15d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:46.861439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef0739-2b8b-47ec-ac5b-9116e121199b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:46.862914 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef0739-2b8b-47ec-ac5b-9116e121199b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d7ef0739-2b8b-47ec-ac5b-9116e121199b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:46.877959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:46.879015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:46.879913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:46.879913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:46.879913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:46.879913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:46.879913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 251/518] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:46.880175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef0739-2b8b-47ec-ac5b-9116e121199b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:46.890180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:46.897010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5917e9-28ee-4003-9877-2320a2eeda38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:46.897631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5917e9-28ee-4003-9877-2320a2eeda38 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:46.897913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5917e9-28ee-4003-9877-2320a2eeda38 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:46.898188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5917e9-28ee-4003-9877-2320a2eeda38 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:46.898915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5917e9-28ee-4003-9877-2320a2eeda38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:46.899225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5917e9-28ee-4003-9877-2320a2eeda38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:46.900602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5917e9-28ee-4003-9877-2320a2eeda38 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:46.913363 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 268/519] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:46.914158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5917e9-28ee-4003-9877-2320a2eeda38 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:46.988680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef0739-2b8b-47ec-ac5b-9116e121199b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:47.013426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5917e9-28ee-4003-9877-2320a2eeda38 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:47.025293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a899ad9-da3c-4754-8451-6f583254a4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:47.025914 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a899ad9-da3c-4754-8451-6f583254a4ae None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:47.026201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a899ad9-da3c-4754-8451-6f583254a4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:47.026475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a899ad9-da3c-4754-8451-6f583254a4ae None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:47.027199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a899ad9-da3c-4754-8451-6f583254a4ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:47.027507 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a899ad9-da3c-4754-8451-6f583254a4ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:47.028892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a899ad9-da3c-4754-8451-6f583254a4ae None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:47.039398 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:47.041453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 252/520] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:47.042332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a899ad9-da3c-4754-8451-6f583254a4ae None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:47.047942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7bee7-91eb-4838-9d5a-4e0c86c5bb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:47.048555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7bee7-91eb-4838-9d5a-4e0c86c5bb99 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:47.048842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7bee7-91eb-4838-9d5a-4e0c86c5bb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:47.049118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7bee7-91eb-4838-9d5a-4e0c86c5bb99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:47.108042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:47.117797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7bee7-91eb-4838-9d5a-4e0c86c5bb99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:47.119295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7bee7-91eb-4838-9d5a-4e0c86c5bb99 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:47.143162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a899ad9-da3c-4754-8451-6f583254a4ae None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:47.177859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc38652-e068-4e10-9b06-0b7adeb3e7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:47.178474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc38652-e068-4e10-9b06-0b7adeb3e7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:47.178756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc38652-e068-4e10-9b06-0b7adeb3e7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:47.179064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc38652-e068-4e10-9b06-0b7adeb3e7b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:47.179812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc38652-e068-4e10-9b06-0b7adeb3e7b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:47.180120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc38652-e068-4e10-9b06-0b7adeb3e7b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:47.181557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc38652-e068-4e10-9b06-0b7adeb3e7b4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:47.198732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 253/521] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:46:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:47.199574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc38652-e068-4e10-9b06-0b7adeb3e7b4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:47.218250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 269/522] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:47.219105 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7bee7-91eb-4838-9d5a-4e0c86c5bb99 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:47.246927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:47.306480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc38652-e068-4e10-9b06-0b7adeb3e7b4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:47.317494 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7bee7-91eb-4838-9d5a-4e0c86c5bb99 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:47.331202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:47.340327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f981a-6e1f-4001-b3af-f48a6271bde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:47.340943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f981a-6e1f-4001-b3af-f48a6271bde8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:47.341228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f981a-6e1f-4001-b3af-f48a6271bde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:47.341503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f981a-6e1f-4001-b3af-f48a6271bde8 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:47.342191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f981a-6e1f-4001-b3af-f48a6271bde8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:47.342489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f981a-6e1f-4001-b3af-f48a6271bde8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:47.343895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f981a-6e1f-4001-b3af-f48a6271bde8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:47.354422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349549d-a16a-48e1-bda4-6b3222621fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:47.355057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349549d-a16a-48e1-bda4-6b3222621fb7 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:47.355342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349549d-a16a-48e1-bda4-6b3222621fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:47.355519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 254/523] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:47.355633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349549d-a16a-48e1-bda4-6b3222621fb7 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:47.356289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349549d-a16a-48e1-bda4-6b3222621fb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:47.356378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f981a-6e1f-4001-b3af-f48a6271bde8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:47.357807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349549d-a16a-48e1-bda4-6b3222621fb7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:47.379422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 270/524] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:46:47 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:46:47.380556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349549d-a16a-48e1-bda4-6b3222621fb7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:47.471315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f981a-6e1f-4001-b3af-f48a6271bde8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:47.492514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:47.511112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722f490-d87e-4b98-a78c-57b490c60359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:47.511875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722f490-d87e-4b98-a78c-57b490c60359 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:47.512236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722f490-d87e-4b98-a78c-57b490c60359 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:47.512653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722f490-d87e-4b98-a78c-57b490c60359 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:47.513525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349549d-a16a-48e1-bda4-6b3222621fb7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:47.521583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5722f490-d87e-4b98-a78c-57b490c60359 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:47.523095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5722f490-d87e-4b98-a78c-57b490c60359 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5722f490-d87e-4b98-a78c-57b490c60359 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:47.535228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:47.537730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 255/525] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:47.538947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722f490-d87e-4b98-a78c-57b490c60359 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:47.562054 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5775e8-4d9b-426e-8813-6adbceef219d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:47.562672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5775e8-4d9b-426e-8813-6adbceef219d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:47.562981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5775e8-4d9b-426e-8813-6adbceef219d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:47.563280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5775e8-4d9b-426e-8813-6adbceef219d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:47.564081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5775e8-4d9b-426e-8813-6adbceef219d None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=162ba16d643747c287bfe8a8e59df235) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:47.564390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5775e8-4d9b-426e-8813-6adbceef219d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:47.565814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5775e8-4d9b-426e-8813-6adbceef219d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:47.574020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 271/526] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:47 2022] GET /identity/v3/role_assignments?user.id=162ba16d643747c287bfe8a8e59df235&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:47.576787 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5775e8-4d9b-426e-8813-6adbceef219d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:47.600866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:47.670218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722f490-d87e-4b98-a78c-57b490c60359 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:47.702615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5775e8-4d9b-426e-8813-6adbceef219d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:47.712081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b0d35-3a81-4776-ad5d-1e0e68e2ec60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:47.713556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b0d35-3a81-4776-ad5d-1e0e68e2ec60 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:47.714077 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b0d35-3a81-4776-ad5d-1e0e68e2ec60 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:47.714669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b0d35-3a81-4776-ad5d-1e0e68e2ec60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:47.727862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b0d35-3a81-4776-ad5d-1e0e68e2ec60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:47.729682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b0d35-3a81-4776-ad5d-1e0e68e2ec60 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-e95b0d35-3a81-4776-ad5d-1e0e68e2ec60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:47.748609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:47.749535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:47.750374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:47.750374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:47.750374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:47.750374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:47.750374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 256/527] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:47.754989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b0d35-3a81-4776-ad5d-1e0e68e2ec60 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:47.755902 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd68f2d-08f8-4580-b4c6-f63bd4d2dbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:47.756562 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd68f2d-08f8-4580-b4c6-f63bd4d2dbff None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:47.756851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd68f2d-08f8-4580-b4c6-f63bd4d2dbff None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:47.757132 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd68f2d-08f8-4580-b4c6-f63bd4d2dbff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:47.768986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd68f2d-08f8-4580-b4c6-f63bd4d2dbff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:47.771506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd68f2d-08f8-4580-b4c6-f63bd4d2dbff None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3cd68f2d-08f8-4580-b4c6-f63bd4d2dbff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:46:47.788028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:47.788917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:47.789875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:46:47.789875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:46:47.789875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:46:47.789875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:46:47.789875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:46:47.789875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:46:47.789875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:47.789875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:47.794028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 272/528] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:47.794814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd68f2d-08f8-4580-b4c6-f63bd4d2dbff None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:47.809982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:47.883600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:47.887899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b0d35-3a81-4776-ad5d-1e0e68e2ec60 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:47.925019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14947aa4-952a-41e1-98a4-b84d53c2c84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:47.925567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14947aa4-952a-41e1-98a4-b84d53c2c84c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:47.925845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14947aa4-952a-41e1-98a4-b84d53c2c84c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:47.926117 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14947aa4-952a-41e1-98a4-b84d53c2c84c None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:47.926730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd68f2d-08f8-4580-b4c6-f63bd4d2dbff None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:47.927304 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14947aa4-952a-41e1-98a4-b84d53c2c84c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:47.927578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14947aa4-952a-41e1-98a4-b84d53c2c84c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:47.929119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14947aa4-952a-41e1-98a4-b84d53c2c84c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:47.957080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 257/529] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Jul 28 03:46:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:47.959118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14947aa4-952a-41e1-98a4-b84d53c2c84c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:47.970322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04891d61-d8f6-4506-87db-9e9763cb4bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:47.970993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04891d61-d8f6-4506-87db-9e9763cb4bad None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:47.971309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04891d61-d8f6-4506-87db-9e9763cb4bad None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:47.971583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04891d61-d8f6-4506-87db-9e9763cb4bad None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:47.972376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04891d61-d8f6-4506-87db-9e9763cb4bad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:47.972682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04891d61-d8f6-4506-87db-9e9763cb4bad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:47.975016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04891d61-d8f6-4506-87db-9e9763cb4bad None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:47.993573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 273/530] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:47.994628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04891d61-d8f6-4506-87db-9e9763cb4bad None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:48.071020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:48.097004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14947aa4-952a-41e1-98a4-b84d53c2c84c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:48.106635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04891d61-d8f6-4506-87db-9e9763cb4bad None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:48.141632 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c4a36-38c0-4cf7-8d11-dc9f8e3a6e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:48.142461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c4a36-38c0-4cf7-8d11-dc9f8e3a6e8e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:48.142811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c4a36-38c0-4cf7-8d11-dc9f8e3a6e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:48.143057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c4a36-38c0-4cf7-8d11-dc9f8e3a6e8e None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:48.143592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe01cd-0c95-4a24-b3b1-f38a79148ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:48.143810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c4a36-38c0-4cf7-8d11-dc9f8e3a6e8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:48.144122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c4a36-38c0-4cf7-8d11-dc9f8e3a6e8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:48.144496 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe01cd-0c95-4a24-b3b1-f38a79148ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:48.144871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe01cd-0c95-4a24-b3b1-f38a79148ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:48.145867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe01cd-0c95-4a24-b3b1-f38a79148ee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:48.146464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c4a36-38c0-4cf7-8d11-dc9f8e3a6e8e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:48.161895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 258/531] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:46:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:48.162601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c4a36-38c0-4cf7-8d11-dc9f8e3a6e8e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:48.187056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:48.229767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe01cd-0c95-4a24-b3b1-f38a79148ee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:48.231602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe01cd-0c95-4a24-b3b1-f38a79148ee6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:48.300335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:48.308726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c4a36-38c0-4cf7-8d11-dc9f8e3a6e8e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:48.348058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 274/532] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:48.349683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe01cd-0c95-4a24-b3b1-f38a79148ee6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:48.356263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bc8fe-d385-4843-97cc-65e04b3d2ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:48.356979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bc8fe-d385-4843-97cc-65e04b3d2ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:48.357334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bc8fe-d385-4843-97cc-65e04b3d2ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:48.357665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bc8fe-d385-4843-97cc-65e04b3d2ff3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:48.365341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bc8fe-d385-4843-97cc-65e04b3d2ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:48.366882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bc8fe-d385-4843-97cc-65e04b3d2ff3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2d9bc8fe-d385-4843-97cc-65e04b3d2ff3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:46:48.378777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:48.381118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:48.383200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:46:48.383200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:46:48.383200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:46:48.383200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:46:48.383200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:46:48.383200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:46:48.383200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:48.383200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:48.386905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 259/533] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:48.386905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bc8fe-d385-4843-97cc-65e04b3d2ff3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:48.467740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe01cd-0c95-4a24-b3b1-f38a79148ee6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:48.478438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:48.503468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bc8fe-d385-4843-97cc-65e04b3d2ff3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:48.511654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14977014-2bb5-4019-9688-61ad01d1a68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:48.512287 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14977014-2bb5-4019-9688-61ad01d1a68c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:48.512570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14977014-2bb5-4019-9688-61ad01d1a68c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:48.512845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14977014-2bb5-4019-9688-61ad01d1a68c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:48.513637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14977014-2bb5-4019-9688-61ad01d1a68c None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=d8967641dd624dacb5ff12c8e22599bd) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:48.513949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14977014-2bb5-4019-9688-61ad01d1a68c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:48.515776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14977014-2bb5-4019-9688-61ad01d1a68c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:48.526817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 275/534] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:48 2022] GET /identity/v3/role_assignments?user.id=d8967641dd624dacb5ff12c8e22599bd&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:48.526817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14977014-2bb5-4019-9688-61ad01d1a68c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:48.566698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b20132-d09b-48b0-8546-82a1616ff174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:48.567921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b20132-d09b-48b0-8546-82a1616ff174 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:48.568404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b20132-d09b-48b0-8546-82a1616ff174 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:48.568908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b20132-d09b-48b0-8546-82a1616ff174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:48.578883 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:48.596736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b20132-d09b-48b0-8546-82a1616ff174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:48.598688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b20132-d09b-48b0-8546-82a1616ff174 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-94b20132-d09b-48b0-8546-82a1616ff174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:48.625555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:48.626474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:48.627335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:48.627335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:48.627335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:48.627335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:48.635248 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 260/535] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:48.641326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b20132-d09b-48b0-8546-82a1616ff174 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:48.664246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14977014-2bb5-4019-9688-61ad01d1a68c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:48.706030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b010ec4-9b8f-4d68-94c9-d11a39d61efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:48.706673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b010ec4-9b8f-4d68-94c9-d11a39d61efa None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:48.706994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b010ec4-9b8f-4d68-94c9-d11a39d61efa None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:48.707274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b010ec4-9b8f-4d68-94c9-d11a39d61efa None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:48.708125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b010ec4-9b8f-4d68-94c9-d11a39d61efa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:48.708450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b010ec4-9b8f-4d68-94c9-d11a39d61efa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:48.709932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b010ec4-9b8f-4d68-94c9-d11a39d61efa None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:48.734723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 276/536] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:46:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:48.743950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b010ec4-9b8f-4d68-94c9-d11a39d61efa None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:48.854920 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b20132-d09b-48b0-8546-82a1616ff174 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:48.913440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:48.929844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37516356-886a-46c4-aa62-9c9378f6d9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:48.930655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37516356-886a-46c4-aa62-9c9378f6d9ca None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:48.932027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37516356-886a-46c4-aa62-9c9378f6d9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:48.932442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37516356-886a-46c4-aa62-9c9378f6d9ca None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:48.936061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37516356-886a-46c4-aa62-9c9378f6d9ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:48.936584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37516356-886a-46c4-aa62-9c9378f6d9ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:48.938088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37516356-886a-46c4-aa62-9c9378f6d9ca None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:48.947669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:48.962667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b010ec4-9b8f-4d68-94c9-d11a39d61efa None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:48.975380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 261/537] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:48.981344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37516356-886a-46c4-aa62-9c9378f6d9ca None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:49.015551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceaf9d-5682-4608-a2d9-5877b57756e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:49.016450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceaf9d-5682-4608-a2d9-5877b57756e9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:49.016872 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceaf9d-5682-4608-a2d9-5877b57756e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:49.017280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceaf9d-5682-4608-a2d9-5877b57756e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:49.018327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceaf9d-5682-4608-a2d9-5877b57756e9 None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=e2cf2558517445c79e47c3b67501ca09) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:49.018774 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceaf9d-5682-4608-a2d9-5877b57756e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:49.021239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceaf9d-5682-4608-a2d9-5877b57756e9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:49.031451 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 277/538] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:48 2022] GET /identity/v3/role_assignments?user.id=e2cf2558517445c79e47c3b67501ca09&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:49.114655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37516356-886a-46c4-aa62-9c9378f6d9ca None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:49.165937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c659a06-48fe-4c8e-9fcf-e907cb45667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:49.166605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659a06-48fe-4c8e-9fcf-e907cb45667d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:49.166956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659a06-48fe-4c8e-9fcf-e907cb45667d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:49.167237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659a06-48fe-4c8e-9fcf-e907cb45667d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:49.175209 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c659a06-48fe-4c8e-9fcf-e907cb45667d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:49.177245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c659a06-48fe-4c8e-9fcf-e907cb45667d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8c659a06-48fe-4c8e-9fcf-e907cb45667d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:46:49.195505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:49.196391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:49.197221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:46:49.197221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:46:49.197221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:46:49.197221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:46:49.197221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:46:49.197221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:46:49.197221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:49.197221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:49.197221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 262/539] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:49.204157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceaf9d-5682-4608-a2d9-5877b57756e9 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:49.383304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceaf9d-5682-4608-a2d9-5877b57756e9 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:49.437376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbafc3c-969a-474d-b89c-95e9d0090f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:49.438005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbafc3c-969a-474d-b89c-95e9d0090f1a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:49.438286 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbafc3c-969a-474d-b89c-95e9d0090f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:49.438563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbafc3c-969a-474d-b89c-95e9d0090f1a None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:49.439737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbafc3c-969a-474d-b89c-95e9d0090f1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:49.440239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbafc3c-969a-474d-b89c-95e9d0090f1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:49.441693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbafc3c-969a-474d-b89c-95e9d0090f1a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:49.458442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 278/540] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:46:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:49.464316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c659a06-48fe-4c8e-9fcf-e907cb45667d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:49.561007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:49.632234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c659a06-48fe-4c8e-9fcf-e907cb45667d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:49.679197 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeb7f4-4b7e-42b2-8793-179f402ff240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:49.679742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb7f4-4b7e-42b2-8793-179f402ff240 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:49.680028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb7f4-4b7e-42b2-8793-179f402ff240 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:49.680304 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb7f4-4b7e-42b2-8793-179f402ff240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:49.681099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeb7f4-4b7e-42b2-8793-179f402ff240 None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=6e31408bfea4412d8f68aea9b3c7528d) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:49.681410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeb7f4-4b7e-42b2-8793-179f402ff240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:49.682873 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeb7f4-4b7e-42b2-8793-179f402ff240 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:49.691229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 263/541] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:49 2022] GET /identity/v3/role_assignments?user.id=6e31408bfea4412d8f68aea9b3c7528d&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:49.830047 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288f087-64bc-49cb-99e6-19874a42b0e3 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:49.830390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288f087-64bc-49cb-99e6-19874a42b0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:49.830604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288f087-64bc-49cb-99e6-19874a42b0e3 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:49.832354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 279/542] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:49.838366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e52de-cf03-4983-9204-abe3206a3c7a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:49.838654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e52de-cf03-4983-9204-abe3206a3c7a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:49.838981 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e52de-cf03-4983-9204-abe3206a3c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:49.847399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8a091-7244-4841-8798-8511ad726051 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:49.847694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8a091-7244-4841-8798-8511ad726051 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:49.847948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8a091-7244-4841-8798-8511ad726051 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:49.850162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 280/543] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:49.864680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5f21f-2517-4799-8c1f-61c39c580748 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:49.865186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5f21f-2517-4799-8c1f-61c39c580748 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:49.865462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5f21f-2517-4799-8c1f-61c39c580748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:49.873058 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b56e52de-cf03-4983-9204-abe3206a3c7a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:49.874275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b56e52de-cf03-4983-9204-abe3206a3c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:49.899994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-80c5f21f-2517-4799-8c1f-61c39c580748 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:49.901271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-80c5f21f-2517-4799-8c1f-61c39c580748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:49.923158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:49.970479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 264/544] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:49.993155 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e52de-cf03-4983-9204-abe3206a3c7a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:50.043582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 281/545] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 03:46:50.062138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:50.084673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5f21f-2517-4799-8c1f-61c39c580748 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:50.160743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e52de-cf03-4983-9204-abe3206a3c7a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:50.217117 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d15735-3c1b-45e8-b1d3-2f93b31d3ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:50.218058 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d15735-3c1b-45e8-b1d3-2f93b31d3ddb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:50.218353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d15735-3c1b-45e8-b1d3-2f93b31d3ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:50.218607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d15735-3c1b-45e8-b1d3-2f93b31d3ddb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:50.242353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d15735-3c1b-45e8-b1d3-2f93b31d3ddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-35d15735-3c1b-45e8-b1d3-2f93b31d3ddb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:50.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:50.249170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:50.250104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 265/546] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:50.257328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d15735-3c1b-45e8-b1d3-2f93b31d3ddb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:50.273387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5f21f-2517-4799-8c1f-61c39c580748 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:50.350648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fd25e-9c96-4ee6-af0c-4d06cd903772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:50.353280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fd25e-9c96-4ee6-af0c-4d06cd903772 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:50.354160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fd25e-9c96-4ee6-af0c-4d06cd903772 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:50.355637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fd25e-9c96-4ee6-af0c-4d06cd903772 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:50.375306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fd25e-9c96-4ee6-af0c-4d06cd903772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-136fd25e-9c96-4ee6-af0c-4d06cd903772 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:50.387226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:50.388333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:50.389319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:50.392717 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 282/547] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:50.398580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fd25e-9c96-4ee6-af0c-4d06cd903772 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:50.412286 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:50.440807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d15735-3c1b-45e8-b1d3-2f93b31d3ddb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:50.443064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:50.518915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d124d6b-0415-471d-9e9b-eb5edbbe3162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:50.519453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d124d6b-0415-471d-9e9b-eb5edbbe3162 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:50.519732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d124d6b-0415-471d-9e9b-eb5edbbe3162 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:50.520026 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d124d6b-0415-471d-9e9b-eb5edbbe3162 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:50.520794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d124d6b-0415-471d-9e9b-eb5edbbe3162 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:50.521095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d124d6b-0415-471d-9e9b-eb5edbbe3162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:50.522473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d124d6b-0415-471d-9e9b-eb5edbbe3162 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:50.547731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 266/548] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:50.553421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d124d6b-0415-471d-9e9b-eb5edbbe3162 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:50.602907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fd25e-9c96-4ee6-af0c-4d06cd903772 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:50.667797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883cbb3-51b8-4051-b022-4325cf282e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:50.668417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883cbb3-51b8-4051-b022-4325cf282e36 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:50.668695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883cbb3-51b8-4051-b022-4325cf282e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:50.668966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883cbb3-51b8-4051-b022-4325cf282e36 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:50.669684 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883cbb3-51b8-4051-b022-4325cf282e36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:50.669986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883cbb3-51b8-4051-b022-4325cf282e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:50.671391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883cbb3-51b8-4051-b022-4325cf282e36 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:50.692617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 283/549] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:46:50.698865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883cbb3-51b8-4051-b022-4325cf282e36 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:50.777650 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d124d6b-0415-471d-9e9b-eb5edbbe3162 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:50.784653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:50.825208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883cbb3-51b8-4051-b022-4325cf282e36 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:50.868032 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb3ffd-c118-4297-bd62-428bc7575501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:50.868936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb3ffd-c118-4297-bd62-428bc7575501 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:50.870292 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb3ffd-c118-4297-bd62-428bc7575501 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:50.871002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb3ffd-c118-4297-bd62-428bc7575501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:50.883305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe47cf-049a-4c98-a086-3602ff5cc8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:50.889252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe47cf-049a-4c98-a086-3602ff5cc8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:50.889682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe47cf-049a-4c98-a086-3602ff5cc8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:50.890116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe47cf-049a-4c98-a086-3602ff5cc8e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:50.903410 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb3ffd-c118-4297-bd62-428bc7575501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:50.905551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb3ffd-c118-4297-bd62-428bc7575501 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:50.912923 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe47cf-049a-4c98-a086-3602ff5cc8e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:50.914719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe47cf-049a-4c98-a086-3602ff5cc8e5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-91bb3ffd-c118-4297-bd62-428bc7575501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:50.922965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:50.924148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:50.925193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:50.925193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:50.925193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:50.925193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:50.927435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 284/550] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:50.932917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb3ffd-c118-4297-bd62-428bc7575501 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-e8fe47cf-049a-4c98-a086-3602ff5cc8e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:50.944916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:50.945841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:50.946701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:50.946701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:50.946701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:50.946701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:50.955036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 267/551] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:50.955990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe47cf-049a-4c98-a086-3602ff5cc8e5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:50.975956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:51.052901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb3ffd-c118-4297-bd62-428bc7575501 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:51.065385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe47cf-049a-4c98-a086-3602ff5cc8e5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:51.074314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:51.089735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76de859-2b9a-4eda-bc69-ee06121709a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:51.090357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76de859-2b9a-4eda-bc69-ee06121709a3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.090699 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76de859-2b9a-4eda-bc69-ee06121709a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.090943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76de859-2b9a-4eda-bc69-ee06121709a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.091640 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76de859-2b9a-4eda-bc69-ee06121709a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:51.091964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76de859-2b9a-4eda-bc69-ee06121709a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:51.093317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76de859-2b9a-4eda-bc69-ee06121709a3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:51.103836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21d1f2-78ac-46e7-9120-b02d5893c1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:51.104699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21d1f2-78ac-46e7-9120-b02d5893c1ac None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.105154 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21d1f2-78ac-46e7-9120-b02d5893c1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.105600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21d1f2-78ac-46e7-9120-b02d5893c1ac None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.106487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21d1f2-78ac-46e7-9120-b02d5893c1ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:51.106592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 285/552] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:51.107083 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21d1f2-78ac-46e7-9120-b02d5893c1ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:51.107855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c40caf-31c9-4384-98c9-87be21dab6f8 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.108147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c40caf-31c9-4384-98c9-87be21dab6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.108424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c40caf-31c9-4384-98c9-87be21dab6f8 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.109191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21d1f2-78ac-46e7-9120-b02d5893c1ac None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:51.109876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 286/553] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:51.111574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c40caf-31c9-4384-98c9-87be21dab6f8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:51.123733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 268/554] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:51.125445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dd1d4-eb1f-42d9-910b-e01471ae61b9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.125950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dd1d4-eb1f-42d9-910b-e01471ae61b9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.126442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dd1d4-eb1f-42d9-910b-e01471ae61b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.145857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-142dd1d4-eb1f-42d9-910b-e01471ae61b9 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:51.147356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-142dd1d4-eb1f-42d9-910b-e01471ae61b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:51.218416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 269/555] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:51.223007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142dd1d4-eb1f-42d9-910b-e01471ae61b9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:51.228872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:51.231963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c40caf-31c9-4384-98c9-87be21dab6f8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:51.268520 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1aa56-0c18-4868-b9a1-ff97590c88da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:51.269513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1aa56-0c18-4868-b9a1-ff97590c88da None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.270193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1aa56-0c18-4868-b9a1-ff97590c88da None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.270640 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1aa56-0c18-4868-b9a1-ff97590c88da None admin] PATH_INFO: `/v3/users/swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.279253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1aa56-0c18-4868-b9a1-ff97590c88da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:51.281387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1aa56-0c18-4868-b9a1-ff97590c88da None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-a9a1aa56-0c18-4868-b9a1-ff97590c88da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:51.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:51.295449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:51.296349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:51.296349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:51.296349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:51.296349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:51.296349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:51.296349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:51.296349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:51.296349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:51.296349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:46:51.296349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:51.296349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 287/556] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:46:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:51.296913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe107-1e36-44c0-8955-b67dc63242e2 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.296913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe107-1e36-44c0-8955-b67dc63242e2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.297030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe107-1e36-44c0-8955-b67dc63242e2 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.304397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 288/557] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:51.306031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfe107-1e36-44c0-8955-b67dc63242e2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:51.334229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142dd1d4-eb1f-42d9-910b-e01471ae61b9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:51.352482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:51.376105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1a155-01d8-4eae-95d4-e4703f93f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:51.376778 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1a155-01d8-4eae-95d4-e4703f93f60d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.377029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1a155-01d8-4eae-95d4-e4703f93f60d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.377296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1a155-01d8-4eae-95d4-e4703f93f60d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.385566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1a155-01d8-4eae-95d4-e4703f93f60d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:51.387052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1a155-01d8-4eae-95d4-e4703f93f60d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-77d1a155-01d8-4eae-95d4-e4703f93f60d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:51.399363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:51.400519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:51.402335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:51.402335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:51.402335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:51.402335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:51.402335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:51.402335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:51.402335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:51.402335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:51.402335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:46:51.402335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:51.405072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 270/558] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jul 28 03:46:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:51.405797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1a155-01d8-4eae-95d4-e4703f93f60d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:51.421432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfe107-1e36-44c0-8955-b67dc63242e2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:51.458943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13256e9f-96c4-4be8-9b0f-222f6d9385da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:51.459617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13256e9f-96c4-4be8-9b0f-222f6d9385da None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.459858 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13256e9f-96c4-4be8-9b0f-222f6d9385da None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.460145 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13256e9f-96c4-4be8-9b0f-222f6d9385da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.470770 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13256e9f-96c4-4be8-9b0f-222f6d9385da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-13256e9f-96c4-4be8-9b0f-222f6d9385da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:51.476436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:51.477372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:51.478183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 289/559] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:51.479440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b5ccf-c887-4e1d-ae4f-1d4d5c896d17 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.479440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b5ccf-c887-4e1d-ae4f-1d4d5c896d17 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.479591 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b5ccf-c887-4e1d-ae4f-1d4d5c896d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.480062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:51.497308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5d0b5ccf-c887-4e1d-ae4f-1d4d5c896d17 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:51.498501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5d0b5ccf-c887-4e1d-ae4f-1d4d5c896d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:51.522116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1a155-01d8-4eae-95d4-e4703f93f60d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:51.564279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc03c3f-6ee7-4227-967e-7d95a1e84283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:51.564825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc03c3f-6ee7-4227-967e-7d95a1e84283 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.565104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc03c3f-6ee7-4227-967e-7d95a1e84283 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.565373 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc03c3f-6ee7-4227-967e-7d95a1e84283 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.566162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc03c3f-6ee7-4227-967e-7d95a1e84283 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:51.566475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc03c3f-6ee7-4227-967e-7d95a1e84283 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:51.568157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc03c3f-6ee7-4227-967e-7d95a1e84283 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:51.582985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 290/560] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:51.582985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b5ccf-c887-4e1d-ae4f-1d4d5c896d17 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:51.594808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 271/561] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jul 28 03:46:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:51.595723 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc03c3f-6ee7-4227-967e-7d95a1e84283 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:51.621629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:51.698205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b5ccf-c887-4e1d-ae4f-1d4d5c896d17 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:51.706692 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc03c3f-6ee7-4227-967e-7d95a1e84283 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:51.734126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:51.738607 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491ca9d-de60-4013-aad4-cee9777833fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:51.739571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ca9d-de60-4013-aad4-cee9777833fb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.739861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ca9d-de60-4013-aad4-cee9777833fb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.740136 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ca9d-de60-4013-aad4-cee9777833fb None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.740943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491ca9d-de60-4013-aad4-cee9777833fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:51.741257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491ca9d-de60-4013-aad4-cee9777833fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:51.742774 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491ca9d-de60-4013-aad4-cee9777833fb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:51.746002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b8e09-4b3b-4a81-8065-8e86df6af6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:51.746830 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b8e09-4b3b-4a81-8065-8e86df6af6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.747302 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b8e09-4b3b-4a81-8065-8e86df6af6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.747746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b8e09-4b3b-4a81-8065-8e86df6af6f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.748625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b8e09-4b3b-4a81-8065-8e86df6af6f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:51.749105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b8e09-4b3b-4a81-8065-8e86df6af6f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:51.750622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b8e09-4b3b-4a81-8065-8e86df6af6f2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:51.760589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 272/562] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:51.761592 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b8e09-4b3b-4a81-8065-8e86df6af6f2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:51.766055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 291/563] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jul 28 03:46:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:51.767031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491ca9d-de60-4013-aad4-cee9777833fb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:51.860366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:51.873447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491ca9d-de60-4013-aad4-cee9777833fb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:51.882201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b8e09-4b3b-4a81-8065-8e86df6af6f2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:51.909506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8ebd8-2329-47dd-b345-84b98d133d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:51.910148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8ebd8-2329-47dd-b345-84b98d133d0b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.910424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8ebd8-2329-47dd-b345-84b98d133d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.911267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8ebd8-2329-47dd-b345-84b98d133d0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.918979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5b888-6cb5-4e28-b3b2-6a3e794ec155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:51.919751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5b888-6cb5-4e28-b3b2-6a3e794ec155 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:51.919860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5b888-6cb5-4e28-b3b2-6a3e794ec155 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:51.920131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5b888-6cb5-4e28-b3b2-6a3e794ec155 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:51.922355 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8ebd8-2329-47dd-b345-84b98d133d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-bfc8ebd8-2329-47dd-b345-84b98d133d0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:51.927690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:51.928613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 292/564] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:51.934297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8ebd8-2329-47dd-b345-84b98d133d0b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:51.985568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:51.999390 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5b888-6cb5-4e28-b3b2-6a3e794ec155 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:52.000843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5b888-6cb5-4e28-b3b2-6a3e794ec155 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:52.032960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8ebd8-2329-47dd-b345-84b98d133d0b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:52.067527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34e11b-58f3-4c3f-9ac6-20f23f8d2004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:52.068241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e11b-58f3-4c3f-9ac6-20f23f8d2004 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.068523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e11b-58f3-4c3f-9ac6-20f23f8d2004 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.068799 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e11b-58f3-4c3f-9ac6-20f23f8d2004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.078925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34e11b-58f3-4c3f-9ac6-20f23f8d2004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:52.080370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34e11b-58f3-4c3f-9ac6-20f23f8d2004 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:52.088905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1e34e11b-58f3-4c3f-9ac6-20f23f8d2004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:52.095668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:52.096668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:52.097589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:52.097589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:52.097589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:52.097589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:52.097589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 293/565] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:52.097854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34e11b-58f3-4c3f-9ac6-20f23f8d2004 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:52.102175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 273/566] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:52.103202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5b888-6cb5-4e28-b3b2-6a3e794ec155 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:52.185344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:52.199129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34e11b-58f3-4c3f-9ac6-20f23f8d2004 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:52.200594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5b888-6cb5-4e28-b3b2-6a3e794ec155 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:52.232939 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c73e73-e7ae-4a3b-8a4f-1e6e3ed9df39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:52.233579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c73e73-e7ae-4a3b-8a4f-1e6e3ed9df39 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.233864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c73e73-e7ae-4a3b-8a4f-1e6e3ed9df39 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.234144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c73e73-e7ae-4a3b-8a4f-1e6e3ed9df39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.235511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d263d-e4f4-47f0-a032-07c14b213ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:52.236192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d263d-e4f4-47f0-a032-07c14b213ecc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.236537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d263d-e4f4-47f0-a032-07c14b213ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.236819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d263d-e4f4-47f0-a032-07c14b213ecc None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.237527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d263d-e4f4-47f0-a032-07c14b213ecc None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:52.237829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d263d-e4f4-47f0-a032-07c14b213ecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:52.239292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d263d-e4f4-47f0-a032-07c14b213ecc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:52.244808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c73e73-e7ae-4a3b-8a4f-1e6e3ed9df39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:52.246248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c73e73-e7ae-4a3b-8a4f-1e6e3ed9df39 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:52.247150 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 274/567] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:52.248306 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8cf1-892a-407e-b168-44894d3d5cd0 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.248591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8cf1-892a-407e-b168-44894d3d5cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.248864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8cf1-892a-407e-b168-44894d3d5cd0 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.250061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 275/568] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:52.250748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8cf1-892a-407e-b168-44894d3d5cd0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-49c73e73-e7ae-4a3b-8a4f-1e6e3ed9df39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:52.262165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:52.263201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:52.264065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:52.264065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:52.264065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:52.264065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:52.264452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 294/569] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:52.265157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c73e73-e7ae-4a3b-8a4f-1e6e3ed9df39 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:52.317103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:52.356947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8cf1-892a-407e-b168-44894d3d5cd0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:52.364875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c73e73-e7ae-4a3b-8a4f-1e6e3ed9df39 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:52.390203 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2ff82-57a4-41be-8619-238637ba4213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:52.390834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2ff82-57a4-41be-8619-238637ba4213 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.391120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2ff82-57a4-41be-8619-238637ba4213 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.391394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2ff82-57a4-41be-8619-238637ba4213 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.392082 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2ff82-57a4-41be-8619-238637ba4213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:52.392389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2ff82-57a4-41be-8619-238637ba4213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:52.393771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2ff82-57a4-41be-8619-238637ba4213 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:52.399078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15a9e1-bac8-4ed3-b5de-270d691cecef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:52.399712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15a9e1-bac8-4ed3-b5de-270d691cecef None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.400000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15a9e1-bac8-4ed3-b5de-270d691cecef None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.400282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15a9e1-bac8-4ed3-b5de-270d691cecef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.405699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 276/570] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:52.406488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2ff82-57a4-41be-8619-238637ba4213 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:52.415437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15a9e1-bac8-4ed3-b5de-270d691cecef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:52.416918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15a9e1-bac8-4ed3-b5de-270d691cecef None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:52.417460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-6e15a9e1-bac8-4ed3-b5de-270d691cecef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:52.432017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:52.432885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:52.433759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:52.433759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:52.433759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:52.433759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:52.434920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 295/571] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:52.436164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81076dfc-1663-4250-8083-8441c62fd000 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.436399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81076dfc-1663-4250-8083-8441c62fd000 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.436680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81076dfc-1663-4250-8083-8441c62fd000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.454419 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-81076dfc-1663-4250-8083-8441c62fd000 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:52.455645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-81076dfc-1663-4250-8083-8441c62fd000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:52.507416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2ff82-57a4-41be-8619-238637ba4213 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:52.521578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 296/572] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:52.522403 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81076dfc-1663-4250-8083-8441c62fd000 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:52.546521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7558a6-ff39-4a55-99b2-ef9d075b3d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:52.547199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7558a6-ff39-4a55-99b2-ef9d075b3d8a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.547486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7558a6-ff39-4a55-99b2-ef9d075b3d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.547763 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7558a6-ff39-4a55-99b2-ef9d075b3d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.558072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7558a6-ff39-4a55-99b2-ef9d075b3d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:52.559547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7558a6-ff39-4a55-99b2-ef9d075b3d8a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:52.561631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-cd7558a6-ff39-4a55-99b2-ef9d075b3d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:52.575176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:52.576060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:52.576955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:52.576955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:52.576955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:52.576955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:52.576955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 277/573] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:52.577222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7558a6-ff39-4a55-99b2-ef9d075b3d8a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:52.631765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81076dfc-1663-4250-8083-8441c62fd000 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:52.637439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:52.665998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888558c3-e0f8-4ab0-9e9b-f224f20651c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:52.666612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888558c3-e0f8-4ab0-9e9b-f224f20651c7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.666972 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888558c3-e0f8-4ab0-9e9b-f224f20651c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.667252 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888558c3-e0f8-4ab0-9e9b-f224f20651c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.667947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888558c3-e0f8-4ab0-9e9b-f224f20651c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:52.668256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888558c3-e0f8-4ab0-9e9b-f224f20651c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:52.669651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888558c3-e0f8-4ab0-9e9b-f224f20651c7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:52.678648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7558a6-ff39-4a55-99b2-ef9d075b3d8a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:52.680875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 297/574] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:52.681668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888558c3-e0f8-4ab0-9e9b-f224f20651c7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:52.713161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9f510-7108-467a-b744-a3c46dbf3b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:52.713730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9f510-7108-467a-b744-a3c46dbf3b20 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.714016 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9f510-7108-467a-b744-a3c46dbf3b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.714293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9f510-7108-467a-b744-a3c46dbf3b20 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.721797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9f510-7108-467a-b744-a3c46dbf3b20 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:52.723895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9f510-7108-467a-b744-a3c46dbf3b20 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-40c9f510-7108-467a-b744-a3c46dbf3b20 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:52.734917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:52.735854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:52.736822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:52.736822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:52.736822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:52.736822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:52.736822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:52.736822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:52.736822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:52.736822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:52.736822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 03:46:52.736822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:52.736822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 278/575] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:52.737403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9f510-7108-467a-b744-a3c46dbf3b20 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:52.778197 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:52.785084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888558c3-e0f8-4ab0-9e9b-f224f20651c7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:52.818705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd331f6-fccf-489e-9835-ad96f5944102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:52.819348 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd331f6-fccf-489e-9835-ad96f5944102 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.819634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd331f6-fccf-489e-9835-ad96f5944102 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.819913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd331f6-fccf-489e-9835-ad96f5944102 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.820611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd331f6-fccf-489e-9835-ad96f5944102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:52.820918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd331f6-fccf-489e-9835-ad96f5944102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:52.822287 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd331f6-fccf-489e-9835-ad96f5944102 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:52.834588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 298/576] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:52.835413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd331f6-fccf-489e-9835-ad96f5944102 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:52.845330 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9f510-7108-467a-b744-a3c46dbf3b20 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:52.866023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:52.880726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbc58e-7c9c-4efd-b333-a01aebdf2e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:52.881363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc58e-7c9c-4efd-b333-a01aebdf2e60 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.881640 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc58e-7c9c-4efd-b333-a01aebdf2e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.881908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc58e-7c9c-4efd-b333-a01aebdf2e60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.892014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbc58e-7c9c-4efd-b333-a01aebdf2e60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-94cbc58e-7c9c-4efd-b333-a01aebdf2e60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:52.898087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:52.899187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 279/577] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:52.900074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbc58e-7c9c-4efd-b333-a01aebdf2e60 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:52.938511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd331f6-fccf-489e-9835-ad96f5944102 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:52.972391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1a11e-e768-4049-835a-6739f68b62c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:52.972935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1a11e-e768-4049-835a-6739f68b62c7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:52.973818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1a11e-e768-4049-835a-6739f68b62c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:52.974180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1a11e-e768-4049-835a-6739f68b62c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:52.974912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1a11e-e768-4049-835a-6739f68b62c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:52.975224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1a11e-e768-4049-835a-6739f68b62c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:52.976601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1a11e-e768-4049-835a-6739f68b62c7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:52.987693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 299/578] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:52.988485 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1a11e-e768-4049-835a-6739f68b62c7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.001061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbc58e-7c9c-4efd-b333-a01aebdf2e60 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.006859 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:53.034150 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f30af-9831-4789-b1b0-583517df1547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.034793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f30af-9831-4789-b1b0-583517df1547 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.035098 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f30af-9831-4789-b1b0-583517df1547 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.035369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f30af-9831-4789-b1b0-583517df1547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.042364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f30af-9831-4789-b1b0-583517df1547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.043851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f30af-9831-4789-b1b0-583517df1547 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c79f30af-9831-4789-b1b0-583517df1547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:46:53.055560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:53.056472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:53.057349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:46:53.057349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:46:53.057349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:46:53.057349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:46:53.057349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:46:53.057349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:46:53.057349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:53.057349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:53.057349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 280/579] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:53.057824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f30af-9831-4789-b1b0-583517df1547 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.095751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1a11e-e768-4049-835a-6739f68b62c7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.101817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:53.129907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7392fe-e56a-4528-829a-ab58c051f236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.130544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7392fe-e56a-4528-829a-ab58c051f236 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.130853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7392fe-e56a-4528-829a-ab58c051f236 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.131135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7392fe-e56a-4528-829a-ab58c051f236 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.131925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7392fe-e56a-4528-829a-ab58c051f236 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:53.132233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7392fe-e56a-4528-829a-ab58c051f236 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.133666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7392fe-e56a-4528-829a-ab58c051f236 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.151466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 300/580] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jul 28 03:46:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:53.152030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7392fe-e56a-4528-829a-ab58c051f236 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.164741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f30af-9831-4789-b1b0-583517df1547 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.198022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c332cd-95cb-4ba1-ba15-8c7620bb0fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.198654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c332cd-95cb-4ba1-ba15-8c7620bb0fca None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.198973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c332cd-95cb-4ba1-ba15-8c7620bb0fca None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.199251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c332cd-95cb-4ba1-ba15-8c7620bb0fca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.205900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c332cd-95cb-4ba1-ba15-8c7620bb0fca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.207376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c332cd-95cb-4ba1-ba15-8c7620bb0fca None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-21c332cd-95cb-4ba1-ba15-8c7620bb0fca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:46:53.219108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:53.219975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:53.220884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:46:53.220884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:46:53.220884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:46:53.220884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:46:53.220884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:46:53.220884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:46:53.220884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:53.220884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:53.220884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 281/581] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:53.221431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c332cd-95cb-4ba1-ba15-8c7620bb0fca None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.226552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:53.256392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7392fe-e56a-4528-829a-ab58c051f236 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.290848 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9c283-4c21-4254-9e08-e474c2a5c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.291467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9c283-4c21-4254-9e08-e474c2a5c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.291754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9c283-4c21-4254-9e08-e474c2a5c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.292031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9c283-4c21-4254-9e08-e474c2a5c1e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.292749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9c283-4c21-4254-9e08-e474c2a5c1e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:53.293053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9c283-4c21-4254-9e08-e474c2a5c1e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.294413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9c283-4c21-4254-9e08-e474c2a5c1e4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.302304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 301/582] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:53.303113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9c283-4c21-4254-9e08-e474c2a5c1e4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.318213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:53.321394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c332cd-95cb-4ba1-ba15-8c7620bb0fca None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.354757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf21f9-5d0e-499d-90dc-98fae89d2b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.355413 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf21f9-5d0e-499d-90dc-98fae89d2b9b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.355693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf21f9-5d0e-499d-90dc-98fae89d2b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.355970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf21f9-5d0e-499d-90dc-98fae89d2b9b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.364322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf21f9-5d0e-499d-90dc-98fae89d2b9b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.365808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf21f9-5d0e-499d-90dc-98fae89d2b9b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7bbf21f9-5d0e-499d-90dc-98fae89d2b9b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:53.377107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:53.378040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:53.378985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:53.378985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:53.378985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:53.378985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:53.378985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:53.378985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:53.378985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:53.378985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:53.378985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 03:46:53.378985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:53.378985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 282/583] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:53.380128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf21f9-5d0e-499d-90dc-98fae89d2b9b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.405504 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9c283-4c21-4254-9e08-e474c2a5c1e4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.438391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35b91a-a13e-4dbf-948a-3aea5884a92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.439032 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35b91a-a13e-4dbf-948a-3aea5884a92f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.439312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35b91a-a13e-4dbf-948a-3aea5884a92f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.439582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35b91a-a13e-4dbf-948a-3aea5884a92f None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.440323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35b91a-a13e-4dbf-948a-3aea5884a92f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:53.440625 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35b91a-a13e-4dbf-948a-3aea5884a92f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.442629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35b91a-a13e-4dbf-948a-3aea5884a92f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.444190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:53.455727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 302/584] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:53.456528 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35b91a-a13e-4dbf-948a-3aea5884a92f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.483926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf21f9-5d0e-499d-90dc-98fae89d2b9b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.516772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5069c8-f85f-4eb8-a6f5-2adbb6727493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.517405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5069c8-f85f-4eb8-a6f5-2adbb6727493 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.517679 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5069c8-f85f-4eb8-a6f5-2adbb6727493 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.517951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5069c8-f85f-4eb8-a6f5-2adbb6727493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.528824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5069c8-f85f-4eb8-a6f5-2adbb6727493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.530262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5069c8-f85f-4eb8-a6f5-2adbb6727493 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1b5069c8-f85f-4eb8-a6f5-2adbb6727493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:53.544832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:53.545718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:53.546600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:53.546600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:53.546600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:53.546600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:53.546600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 283/585] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:53.547380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5069c8-f85f-4eb8-a6f5-2adbb6727493 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.562849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35b91a-a13e-4dbf-948a-3aea5884a92f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.566037 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:53.596879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404fb72-1507-4e66-9a92-b7efb044cf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.597501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404fb72-1507-4e66-9a92-b7efb044cf81 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.597781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404fb72-1507-4e66-9a92-b7efb044cf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.598645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404fb72-1507-4e66-9a92-b7efb044cf81 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.599484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404fb72-1507-4e66-9a92-b7efb044cf81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:53.599794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404fb72-1507-4e66-9a92-b7efb044cf81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.601231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404fb72-1507-4e66-9a92-b7efb044cf81 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.612627 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 303/586] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:53.613251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404fb72-1507-4e66-9a92-b7efb044cf81 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.654485 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5069c8-f85f-4eb8-a6f5-2adbb6727493 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.668468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:53.688720 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79538400-eff8-456d-b805-6d9e91acd86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.689443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538400-eff8-456d-b805-6d9e91acd86d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.689797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538400-eff8-456d-b805-6d9e91acd86d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.690145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538400-eff8-456d-b805-6d9e91acd86d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.700872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79538400-eff8-456d-b805-6d9e91acd86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.702416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79538400-eff8-456d-b805-6d9e91acd86d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.712376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404fb72-1507-4e66-9a92-b7efb044cf81 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-79538400-eff8-456d-b805-6d9e91acd86d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:53.717233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:53.718246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:53.719189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:53.719189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:53.719189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:53.719189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:53.719189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 284/587] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:53.719456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79538400-eff8-456d-b805-6d9e91acd86d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.749564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0393a5-0033-4c72-a5d4-e266d754e710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.750107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0393a5-0033-4c72-a5d4-e266d754e710 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.750382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0393a5-0033-4c72-a5d4-e266d754e710 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.750655 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0393a5-0033-4c72-a5d4-e266d754e710 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.751473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0393a5-0033-4c72-a5d4-e266d754e710 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:53.751778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0393a5-0033-4c72-a5d4-e266d754e710 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.753202 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0393a5-0033-4c72-a5d4-e266d754e710 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.770465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 304/588] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jul 28 03:46:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:53.771494 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0393a5-0033-4c72-a5d4-e266d754e710 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.781645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:53.820817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79538400-eff8-456d-b805-6d9e91acd86d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.853347 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901e168-f5b7-4bfd-a5fb-33c5c207964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.854001 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901e168-f5b7-4bfd-a5fb-33c5c207964c None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.854275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901e168-f5b7-4bfd-a5fb-33c5c207964c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.854552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901e168-f5b7-4bfd-a5fb-33c5c207964c None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/162ba16d643747c287bfe8a8e59df235/roles/f5a9eb03e88a4e66bb9a8b6180009f72` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.868976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901e168-f5b7-4bfd-a5fb-33c5c207964c None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=162ba16d643747c287bfe8a8e59df235, role_id=f5a9eb03e88a4e66bb9a8b6180009f72)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.869132 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0393a5-0033-4c72-a5d4-e266d754e710 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:53.870420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901e168-f5b7-4bfd-a5fb-33c5c207964c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.883221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:53.884434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 285/589] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:46:53 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/162ba16d643747c287bfe8a8e59df235/roles/f5a9eb03e88a4e66bb9a8b6180009f72 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:53.885275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901e168-f5b7-4bfd-a5fb-33c5c207964c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:53.908011 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3a694-fd32-4b11-be2c-98253a0e00e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:53.908828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3a694-fd32-4b11-be2c-98253a0e00e5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:53.909026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3a694-fd32-4b11-be2c-98253a0e00e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:53.909305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3a694-fd32-4b11-be2c-98253a0e00e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:53.910081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3a694-fd32-4b11-be2c-98253a0e00e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:53.910327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3a694-fd32-4b11-be2c-98253a0e00e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:53.913694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3a694-fd32-4b11-be2c-98253a0e00e5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:53.931256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 305/590] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:53.931256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3a694-fd32-4b11-be2c-98253a0e00e5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:54.016214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901e168-f5b7-4bfd-a5fb-33c5c207964c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:54.043076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3a694-fd32-4b11-be2c-98253a0e00e5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:54.043864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:54.068213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1933b23-c537-4707-9829-540e65989ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:54.068829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1933b23-c537-4707-9829-540e65989ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:54.069111 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1933b23-c537-4707-9829-540e65989ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:54.069383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1933b23-c537-4707-9829-540e65989ea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:54.070174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1933b23-c537-4707-9829-540e65989ea2 None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=19779569e08a42148d2933f644e3b9fa) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:54.070482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1933b23-c537-4707-9829-540e65989ea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:54.071953 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1933b23-c537-4707-9829-540e65989ea2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:54.078019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 286/591] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:53 2022] GET /identity/v3/role_assignments?user.id=19779569e08a42148d2933f644e3b9fa&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:54.078740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1933b23-c537-4707-9829-540e65989ea2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:54.090165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a188086-dd87-4ecd-9c28-f40c11b489c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:54.090939 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a188086-dd87-4ecd-9c28-f40c11b489c3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:54.091286 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a188086-dd87-4ecd-9c28-f40c11b489c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:54.091629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a188086-dd87-4ecd-9c28-f40c11b489c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:54.092422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a188086-dd87-4ecd-9c28-f40c11b489c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:54.092800 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a188086-dd87-4ecd-9c28-f40c11b489c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:54.094375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a188086-dd87-4ecd-9c28-f40c11b489c3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:54.108563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 306/592] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:54.109535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a188086-dd87-4ecd-9c28-f40c11b489c3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:54.151957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:54.222947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1933b23-c537-4707-9829-540e65989ea2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:54.236341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a188086-dd87-4ecd-9c28-f40c11b489c3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:54.270409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914cb26-4e72-4a35-9b04-7bba07c7bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:54.272636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914cb26-4e72-4a35-9b04-7bba07c7bcee None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:54.272636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914cb26-4e72-4a35-9b04-7bba07c7bcee None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:54.272636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914cb26-4e72-4a35-9b04-7bba07c7bcee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:54.279278 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28d907-7404-42f3-87d4-ca9e26b95dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:54.279917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28d907-7404-42f3-87d4-ca9e26b95dbd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:54.280199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28d907-7404-42f3-87d4-ca9e26b95dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:54.281342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28d907-7404-42f3-87d4-ca9e26b95dbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:54.291303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914cb26-4e72-4a35-9b04-7bba07c7bcee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:54.291788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28d907-7404-42f3-87d4-ca9e26b95dbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:54.292767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914cb26-4e72-4a35-9b04-7bba07c7bcee None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:54.293858 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28d907-7404-42f3-87d4-ca9e26b95dbd None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2914cb26-4e72-4a35-9b04-7bba07c7bcee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:54.308259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:54.309188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:54.310060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:54.310060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:54.310060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:54.310060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-cc28d907-7404-42f3-87d4-ca9e26b95dbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:46:54.310703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:54.311629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:54.312497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:46:54.312497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:46:54.312497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:46:54.312497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:46:54.312497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:46:54.312497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:46:54.312497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:54.312497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:54.313990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 287/593] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:54.314846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914cb26-4e72-4a35-9b04-7bba07c7bcee None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:54.315203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 307/594] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:54.319622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28d907-7404-42f3-87d4-ca9e26b95dbd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:54.354844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:54.421694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:54.431952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28d907-7404-42f3-87d4-ca9e26b95dbd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:54.434035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914cb26-4e72-4a35-9b04-7bba07c7bcee None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:54.469292 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f28b2d-7a2a-43b2-b380-8f71bafc2e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:54.470372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f28b2d-7a2a-43b2-b380-8f71bafc2e1e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:54.470480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d7e1a-bc6e-4741-aa48-4977ac64996a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:54.471017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f28b2d-7a2a-43b2-b380-8f71bafc2e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:54.471123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7e1a-bc6e-4741-aa48-4977ac64996a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:54.471317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7e1a-bc6e-4741-aa48-4977ac64996a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:54.471591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7e1a-bc6e-4741-aa48-4977ac64996a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:54.471968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f28b2d-7a2a-43b2-b380-8f71bafc2e1e None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:54.473029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f28b2d-7a2a-43b2-b380-8f71bafc2e1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:54.473693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f28b2d-7a2a-43b2-b380-8f71bafc2e1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:54.475314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f28b2d-7a2a-43b2-b380-8f71bafc2e1e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:54.479744 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d7e1a-bc6e-4741-aa48-4977ac64996a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:54.481253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d7e1a-bc6e-4741-aa48-4977ac64996a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:54.492743 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 308/595] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-939d7e1a-bc6e-4741-aa48-4977ac64996a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:54.493698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:54.494691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:54.495697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:54.495697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:54.495697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:54.495697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:54.495697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:54.495697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:54.495697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:54.495697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:54.495697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 03:46:54.495697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:54.495697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 288/596] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:54.496271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d7e1a-bc6e-4741-aa48-4977ac64996a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:54.496687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f28b2d-7a2a-43b2-b380-8f71bafc2e1e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:54.601167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:54.606434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f28b2d-7a2a-43b2-b380-8f71bafc2e1e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:54.615322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d7e1a-bc6e-4741-aa48-4977ac64996a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:54.644767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24656498-8a93-49bb-9d0e-6f8cfbaa93c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:54.645792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24656498-8a93-49bb-9d0e-6f8cfbaa93c1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:54.646217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24656498-8a93-49bb-9d0e-6f8cfbaa93c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:54.646685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24656498-8a93-49bb-9d0e-6f8cfbaa93c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:54.647628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24656498-8a93-49bb-9d0e-6f8cfbaa93c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:54.648079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24656498-8a93-49bb-9d0e-6f8cfbaa93c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:54.649654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24656498-8a93-49bb-9d0e-6f8cfbaa93c1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:54.658031 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e22925-4082-4c55-8d07-6d44a7ee2346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:54.659510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22925-4082-4c55-8d07-6d44a7ee2346 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:54.659795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22925-4082-4c55-8d07-6d44a7ee2346 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:54.660066 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22925-4082-4c55-8d07-6d44a7ee2346 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:54.671256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 309/597] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:46:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:54.671256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24656498-8a93-49bb-9d0e-6f8cfbaa93c1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:54.675418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e22925-4082-4c55-8d07-6d44a7ee2346 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:54.677665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e22925-4082-4c55-8d07-6d44a7ee2346 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-74e22925-4082-4c55-8d07-6d44a7ee2346 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:46:54.689358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:54.690364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:54.691904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:46:54.691904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:46:54.691904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:46:54.691904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:46:54.691904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:46:54.691904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:46:54.691904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:54.691904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:54.694220 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:54.695032 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 289/598] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:54.695712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e22925-4082-4c55-8d07-6d44a7ee2346 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:54.807294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e22925-4082-4c55-8d07-6d44a7ee2346 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:54.808014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24656498-8a93-49bb-9d0e-6f8cfbaa93c1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:54.844703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a633910-9553-4242-82b3-fd853fd0eb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:54.845365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a633910-9553-4242-82b3-fd853fd0eb2c None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:54.845645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a633910-9553-4242-82b3-fd853fd0eb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:54.845919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a633910-9553-4242-82b3-fd853fd0eb2c None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/d8967641dd624dacb5ff12c8e22599bd/roles/f5a9eb03e88a4e66bb9a8b6180009f72` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:54.855583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89d227-21e6-43e5-96ec-2ca8c97fb95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:54.856771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89d227-21e6-43e5-96ec-2ca8c97fb95a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:54.857267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89d227-21e6-43e5-96ec-2ca8c97fb95a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:54.857763 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89d227-21e6-43e5-96ec-2ca8c97fb95a None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:54.859276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89d227-21e6-43e5-96ec-2ca8c97fb95a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:54.860064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89d227-21e6-43e5-96ec-2ca8c97fb95a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:54.861646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a633910-9553-4242-82b3-fd853fd0eb2c None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=d8967641dd624dacb5ff12c8e22599bd, role_id=f5a9eb03e88a4e66bb9a8b6180009f72)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:54.862175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89d227-21e6-43e5-96ec-2ca8c97fb95a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:54.863185 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a633910-9553-4242-82b3-fd853fd0eb2c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:54.875613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:54.890517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 290/599] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:46:54 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/d8967641dd624dacb5ff12c8e22599bd/roles/f5a9eb03e88a4e66bb9a8b6180009f72 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:54.890517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a633910-9553-4242-82b3-fd853fd0eb2c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:54.891074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 310/600] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Jul 28 03:46:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:54.900504 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89d227-21e6-43e5-96ec-2ca8c97fb95a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:55.011705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a633910-9553-4242-82b3-fd853fd0eb2c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:55.050654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87aa70-5965-4a4c-a07d-ca1a094657aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:55.051906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87aa70-5965-4a4c-a07d-ca1a094657aa None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:55.052123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87aa70-5965-4a4c-a07d-ca1a094657aa None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:55.052399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87aa70-5965-4a4c-a07d-ca1a094657aa None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:55.053160 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87aa70-5965-4a4c-a07d-ca1a094657aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:55.053467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87aa70-5965-4a4c-a07d-ca1a094657aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:55.054963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87aa70-5965-4a4c-a07d-ca1a094657aa None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:55.055497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:55.068576 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 291/601] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:46:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:55.074967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87aa70-5965-4a4c-a07d-ca1a094657aa None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:55.125304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89d227-21e6-43e5-96ec-2ca8c97fb95a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:55.169926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c067b-a49c-4c9d-8d9f-b78b986331b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:55.170979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c067b-a49c-4c9d-8d9f-b78b986331b9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:55.171472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c067b-a49c-4c9d-8d9f-b78b986331b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:55.171949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c067b-a49c-4c9d-8d9f-b78b986331b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:55.175381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:55.185302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c067b-a49c-4c9d-8d9f-b78b986331b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:55.187067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c067b-a49c-4c9d-8d9f-b78b986331b9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-476c067b-a49c-4c9d-8d9f-b78b986331b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:55.204293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:55.205227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:55.206080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:55.206080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:55.206080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:55.206080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:55.209560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 311/602] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:55.212573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c067b-a49c-4c9d-8d9f-b78b986331b9 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:55.240347 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87aa70-5965-4a4c-a07d-ca1a094657aa None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:55.305421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137e4d5-1adc-4b3d-8f7c-3ac96d1c56d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:55.306464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137e4d5-1adc-4b3d-8f7c-3ac96d1c56d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:55.306945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137e4d5-1adc-4b3d-8f7c-3ac96d1c56d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:55.307386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137e4d5-1adc-4b3d-8f7c-3ac96d1c56d6 None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/e2cf2558517445c79e47c3b67501ca09/roles/f5a9eb03e88a4e66bb9a8b6180009f72` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:55.318533 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c067b-a49c-4c9d-8d9f-b78b986331b9 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:55.340639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137e4d5-1adc-4b3d-8f7c-3ac96d1c56d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=e2cf2558517445c79e47c3b67501ca09, role_id=f5a9eb03e88a4e66bb9a8b6180009f72)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:55.342476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137e4d5-1adc-4b3d-8f7c-3ac96d1c56d6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:55.344570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:55.359968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f8cc3-f7b5-4245-bc0e-fbe5cba33bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:55.360817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f8cc3-f7b5-4245-bc0e-fbe5cba33bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:55.361185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f8cc3-f7b5-4245-bc0e-fbe5cba33bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:55.361581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f8cc3-f7b5-4245-bc0e-fbe5cba33bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:55.362358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f8cc3-f7b5-4245-bc0e-fbe5cba33bd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:55.362757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f8cc3-f7b5-4245-bc0e-fbe5cba33bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:55.365166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f8cc3-f7b5-4245-bc0e-fbe5cba33bd7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:55.375687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 292/603] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:46:55 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/e2cf2558517445c79e47c3b67501ca09/roles/f5a9eb03e88a4e66bb9a8b6180009f72 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:55.382882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 312/604] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:55.387956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137e4d5-1adc-4b3d-8f7c-3ac96d1c56d6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:55.506423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137e4d5-1adc-4b3d-8f7c-3ac96d1c56d6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:55.518999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:55.575644 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b4e5f-73cf-479a-90c2-6d1bb3afa577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:55.576749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b4e5f-73cf-479a-90c2-6d1bb3afa577 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:55.577226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b4e5f-73cf-479a-90c2-6d1bb3afa577 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:55.577707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b4e5f-73cf-479a-90c2-6d1bb3afa577 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:55.589978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b4e5f-73cf-479a-90c2-6d1bb3afa577 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:55.591872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b4e5f-73cf-479a-90c2-6d1bb3afa577 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ae1b4e5f-73cf-479a-90c2-6d1bb3afa577 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:46:55.605808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:55.606690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:55.607601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:46:55.607601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:46:55.607601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:46:55.607601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:46:55.607601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:46:55.607601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:46:55.607601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:55.607601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:55.610041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 293/605] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:55.622825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f8cc3-f7b5-4245-bc0e-fbe5cba33bd7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:55.734880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f8cc3-f7b5-4245-bc0e-fbe5cba33bd7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:55.772638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04cec5-793c-445a-b771-df52468b221e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:55.773505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04cec5-793c-445a-b771-df52468b221e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:55.773978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04cec5-793c-445a-b771-df52468b221e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:55.774403 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04cec5-793c-445a-b771-df52468b221e None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:55.775336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04cec5-793c-445a-b771-df52468b221e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:55.775792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04cec5-793c-445a-b771-df52468b221e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:55.777388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04cec5-793c-445a-b771-df52468b221e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:55.794889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 313/606] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:55.796791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b4e5f-73cf-479a-90c2-6d1bb3afa577 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:55.910322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b4e5f-73cf-479a-90c2-6d1bb3afa577 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:55.958902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1619127-688e-4c89-bf5a-db512ff162e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:55.959827 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1619127-688e-4c89-bf5a-db512ff162e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:55.960286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1619127-688e-4c89-bf5a-db512ff162e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:55.960709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1619127-688e-4c89-bf5a-db512ff162e0 None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/6e31408bfea4412d8f68aea9b3c7528d/roles/f5a9eb03e88a4e66bb9a8b6180009f72` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:55.982706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1619127-688e-4c89-bf5a-db512ff162e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=6e31408bfea4412d8f68aea9b3c7528d, role_id=f5a9eb03e88a4e66bb9a8b6180009f72)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:55.984491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1619127-688e-4c89-bf5a-db512ff162e0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:56.006145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 294/607] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:46:55 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/6e31408bfea4412d8f68aea9b3c7528d/roles/f5a9eb03e88a4e66bb9a8b6180009f72 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:46:56.020906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87722551-a70c-4f32-9747-aba659efe054 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:56.021477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87722551-a70c-4f32-9747-aba659efe054 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:56.021907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87722551-a70c-4f32-9747-aba659efe054 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:56.023648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 314/608] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:56.029403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b61e6-5485-4768-a754-6f8bbe77b434 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:56.029909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b61e6-5485-4768-a754-6f8bbe77b434 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:56.030361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b61e6-5485-4768-a754-6f8bbe77b434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:56.056334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2a1b61e6-5485-4768-a754-6f8bbe77b434 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:56.058312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2a1b61e6-5485-4768-a754-6f8bbe77b434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:56.080398 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:56.157484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 295/609] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:56.189047 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87722551-a70c-4f32-9747-aba659efe054 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:56.243401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:56.312554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87722551-a70c-4f32-9747-aba659efe054 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:56.348683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cb7ad-9e21-4f47-ae94-d96b3e702a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:56.349330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cb7ad-9e21-4f47-ae94-d96b3e702a83 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:56.349616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cb7ad-9e21-4f47-ae94-d96b3e702a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:56.349893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cb7ad-9e21-4f47-ae94-d96b3e702a83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:56.360738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cb7ad-9e21-4f47-ae94-d96b3e702a83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-838cb7ad-9e21-4f47-ae94-d96b3e702a83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:56.366605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:56.367579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:56.368409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 315/610] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:56.373643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b61e6-5485-4768-a754-6f8bbe77b434 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:56.493010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b61e6-5485-4768-a754-6f8bbe77b434 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:56.521247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df70a2-e2b5-4676-806b-ff661a2cd951 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:56.521766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df70a2-e2b5-4676-806b-ff661a2cd951 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:56.522215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df70a2-e2b5-4676-806b-ff661a2cd951 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:56.523993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 316/611] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:56.528980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933aa004-3d08-4935-8a97-63ba3460d6c3 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:56.529406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933aa004-3d08-4935-8a97-63ba3460d6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:56.529792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933aa004-3d08-4935-8a97-63ba3460d6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:56.530531 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489b277-164e-4606-abf4-2fd68f9b0ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:56.531678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489b277-164e-4606-abf4-2fd68f9b0ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:56.531964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489b277-164e-4606-abf4-2fd68f9b0ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:56.532238 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489b277-164e-4606-abf4-2fd68f9b0ac5 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:56.533048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489b277-164e-4606-abf4-2fd68f9b0ac5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:56.533362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489b277-164e-4606-abf4-2fd68f9b0ac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:56.534719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489b277-164e-4606-abf4-2fd68f9b0ac5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:56.545049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 296/612] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:56.553375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489b277-164e-4606-abf4-2fd68f9b0ac5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:56.571392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:56.583871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-933aa004-3d08-4935-8a97-63ba3460d6c3 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:56.585100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-933aa004-3d08-4935-8a97-63ba3460d6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:56.684299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489b277-164e-4606-abf4-2fd68f9b0ac5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:56.718191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9869cb-00d1-48d6-be51-a3cdee91065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:56.718847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9869cb-00d1-48d6-be51-a3cdee91065c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:56.719133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9869cb-00d1-48d6-be51-a3cdee91065c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:56.719406 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9869cb-00d1-48d6-be51-a3cdee91065c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:56.729957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9869cb-00d1-48d6-be51-a3cdee91065c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:56.731433 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9869cb-00d1-48d6-be51-a3cdee91065c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:56.735878 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 317/613] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7c9869cb-00d1-48d6-be51-a3cdee91065c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:56.746685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:56.747754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:56.748797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:56.748797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:56.748797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:56.748797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:56.748797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 297/614] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:56.753186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933aa004-3d08-4935-8a97-63ba3460d6c3 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:56.762819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9869cb-00d1-48d6-be51-a3cdee91065c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:56.822301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:56.838940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:56.903041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9869cb-00d1-48d6-be51-a3cdee91065c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:56.972561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933aa004-3d08-4935-8a97-63ba3460d6c3 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:56.975919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f87ae-c8c0-4781-9f04-dc0f1166cee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:56.976572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f87ae-c8c0-4781-9f04-dc0f1166cee6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:56.976848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f87ae-c8c0-4781-9f04-dc0f1166cee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:56.977114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f87ae-c8c0-4781-9f04-dc0f1166cee6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:57.002116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f87ae-c8c0-4781-9f04-dc0f1166cee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1c9f87ae-c8c0-4781-9f04-dc0f1166cee6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:57.010904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:57.011915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:57.012746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 298/615] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:57.017920 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f87ae-c8c0-4781-9f04-dc0f1166cee6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:57.063717 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eae3f8-a27f-4523-a0bf-f181b20a5cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:57.064339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae3f8-a27f-4523-a0bf-f181b20a5cbf None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:57.064617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae3f8-a27f-4523-a0bf-f181b20a5cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:57.064889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae3f8-a27f-4523-a0bf-f181b20a5cbf None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:57.065580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eae3f8-a27f-4523-a0bf-f181b20a5cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:57.065879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eae3f8-a27f-4523-a0bf-f181b20a5cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:57.067824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eae3f8-a27f-4523-a0bf-f181b20a5cbf None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:57.095908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 318/616] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:57.102072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eae3f8-a27f-4523-a0bf-f181b20a5cbf None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:57.175411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f87ae-c8c0-4781-9f04-dc0f1166cee6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:57.203176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:57.214032 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ef197-115d-4fcd-8090-0cf16949b109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:57.214665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ef197-115d-4fcd-8090-0cf16949b109 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:57.215370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ef197-115d-4fcd-8090-0cf16949b109 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:57.215690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ef197-115d-4fcd-8090-0cf16949b109 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:57.216456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ef197-115d-4fcd-8090-0cf16949b109 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:57.216816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ef197-115d-4fcd-8090-0cf16949b109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:57.218309 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ef197-115d-4fcd-8090-0cf16949b109 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:57.229690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 299/617] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:57.238861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ef197-115d-4fcd-8090-0cf16949b109 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:57.336859 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:57.357755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eae3f8-a27f-4523-a0bf-f181b20a5cbf None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:57.364671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ef197-115d-4fcd-8090-0cf16949b109 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:57.425902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c515ba9-091a-4db4-9916-57ca735c7b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:57.428316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c515ba9-091a-4db4-9916-57ca735c7b16 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:57.429572 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92175b8d-5e2d-4d7a-acd4-a635698a4554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:57.430010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c515ba9-091a-4db4-9916-57ca735c7b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:57.430451 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c515ba9-091a-4db4-9916-57ca735c7b16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:57.431281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92175b8d-5e2d-4d7a-acd4-a635698a4554 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:57.433801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92175b8d-5e2d-4d7a-acd4-a635698a4554 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:57.434224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92175b8d-5e2d-4d7a-acd4-a635698a4554 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:57.450601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92175b8d-5e2d-4d7a-acd4-a635698a4554 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:57.451662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c515ba9-091a-4db4-9916-57ca735c7b16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:57.453245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92175b8d-5e2d-4d7a-acd4-a635698a4554 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:57.454813 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c515ba9-091a-4db4-9916-57ca735c7b16 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-92175b8d-5e2d-4d7a-acd4-a635698a4554 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:57.477164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:57.478371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:57.479631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:57.479631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:57.479631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:57.479631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:57.479631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:57.479631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:57.479631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:57.479631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:57.479631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:46:57.479631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6c515ba9-091a-4db4-9916-57ca735c7b16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:57.486602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:57.487689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:57.489757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:57.489757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:57.489757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:57.489757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:57.491661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 319/618] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jul 28 03:46:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:46:57.493283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 300/619] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:57.499956 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92175b8d-5e2d-4d7a-acd4-a635698a4554 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:57.503790 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c515ba9-091a-4db4-9916-57ca735c7b16 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:57.591282 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:57.656616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92175b8d-5e2d-4d7a-acd4-a635698a4554 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:57.668356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c515ba9-091a-4db4-9916-57ca735c7b16 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:57.702707 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3af22-f55b-4c6f-bf35-6eaaeac42c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:57.703761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3af22-f55b-4c6f-bf35-6eaaeac42c0f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:57.704385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3af22-f55b-4c6f-bf35-6eaaeac42c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:57.704670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3af22-f55b-4c6f-bf35-6eaaeac42c0f None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:57.705730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3af22-f55b-4c6f-bf35-6eaaeac42c0f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:57.706042 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3af22-f55b-4c6f-bf35-6eaaeac42c0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:57.709629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3af22-f55b-4c6f-bf35-6eaaeac42c0f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:57.712456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa61c4-1fb5-440b-b34e-f8c8e213f0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:57.714025 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa61c4-1fb5-440b-b34e-f8c8e213f0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:57.714459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa61c4-1fb5-440b-b34e-f8c8e213f0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:57.714864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa61c4-1fb5-440b-b34e-f8c8e213f0d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:57.715648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa61c4-1fb5-440b-b34e-f8c8e213f0d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:57.716030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa61c4-1fb5-440b-b34e-f8c8e213f0d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:57.717491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa61c4-1fb5-440b-b34e-f8c8e213f0d3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:57.732426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 320/620] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jul 28 03:46:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:57.733104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 301/621] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:57.738063 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8c6a7-af4f-4d05-810e-8221de4983ab None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:57.738366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8c6a7-af4f-4d05-810e-8221de4983ab None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:57.739005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8c6a7-af4f-4d05-810e-8221de4983ab None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:57.739122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa61c4-1fb5-440b-b34e-f8c8e213f0d3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:57.740864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 321/622] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:57.741592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8c6a7-af4f-4d05-810e-8221de4983ab None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:57.759687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:57.861601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:57.866112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa61c4-1fb5-440b-b34e-f8c8e213f0d3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:57.878325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8c6a7-af4f-4d05-810e-8221de4983ab None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:57.904270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf3d9a-148f-4a9a-836b-fe02de0df3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:57.904896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf3d9a-148f-4a9a-836b-fe02de0df3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:57.905241 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf3d9a-148f-4a9a-836b-fe02de0df3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:57.905452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf3d9a-148f-4a9a-836b-fe02de0df3c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:57.923356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73b6c7-e0e5-478a-be41-7c113e53c9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:57.924267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73b6c7-e0e5-478a-be41-7c113e53c9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:57.924710 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73b6c7-e0e5-478a-be41-7c113e53c9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:57.925133 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73b6c7-e0e5-478a-be41-7c113e53c9c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:57.934026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73b6c7-e0e5-478a-be41-7c113e53c9c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:57.935807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73b6c7-e0e5-478a-be41-7c113e53c9c3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-0e73b6c7-e0e5-478a-be41-7c113e53c9c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:57.949783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:57.950991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:57.952094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:57.952094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:57.952094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:57.952094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:57.952094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:57.952094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:57.952094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:57.952094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:57.952094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:46:57.952094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:57.954258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 322/623] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:46:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:57.955604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039efe6-3cfd-4520-9a83-6462eedb6a7d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:57.956123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039efe6-3cfd-4520-9a83-6462eedb6a7d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:57.956568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039efe6-3cfd-4520-9a83-6462eedb6a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:57.978745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5039efe6-3cfd-4520-9a83-6462eedb6a7d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:57.980205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5039efe6-3cfd-4520-9a83-6462eedb6a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:57.984225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf3d9a-148f-4a9a-836b-fe02de0df3c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:57.985686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf3d9a-148f-4a9a-836b-fe02de0df3c0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:58.053994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:58.058805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 323/624] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:58.059603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039efe6-3cfd-4520-9a83-6462eedb6a7d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:58.092017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:58.102840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 302/625] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:46:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:46:58.102840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f684e-82ef-4f3e-bd90-43b08ab94543 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.102840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f684e-82ef-4f3e-bd90-43b08ab94543 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.102840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f684e-82ef-4f3e-bd90-43b08ab94543 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.102840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 303/626] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:58.102840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f684e-82ef-4f3e-bd90-43b08ab94543 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:58.176953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039efe6-3cfd-4520-9a83-6462eedb6a7d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:58.212386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602af90d-027c-4d6a-949f-48601483d9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:58.213120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602af90d-027c-4d6a-949f-48601483d9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.213429 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602af90d-027c-4d6a-949f-48601483d9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.213742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602af90d-027c-4d6a-949f-48601483d9e7 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.214397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f684e-82ef-4f3e-bd90-43b08ab94543 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:58.214978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602af90d-027c-4d6a-949f-48601483d9e7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:58.214978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602af90d-027c-4d6a-949f-48601483d9e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:58.216563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602af90d-027c-4d6a-949f-48601483d9e7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:58.240288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 324/627] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jul 28 03:46:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:58.240288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602af90d-027c-4d6a-949f-48601483d9e7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:58.253706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73cbec9-cc37-40b3-b5fc-6496f77c05ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:58.254542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cbec9-cc37-40b3-b5fc-6496f77c05ec None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.255042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cbec9-cc37-40b3-b5fc-6496f77c05ec None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.255596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cbec9-cc37-40b3-b5fc-6496f77c05ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.266283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73cbec9-cc37-40b3-b5fc-6496f77c05ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d73cbec9-cc37-40b3-b5fc-6496f77c05ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:58.271998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:58.272922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:58.273741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 304/628] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:58.274434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97235b-626b-4257-bdb8-91066793dcc2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.274858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97235b-626b-4257-bdb8-91066793dcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.274966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97235b-626b-4257-bdb8-91066793dcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.293705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5d97235b-626b-4257-bdb8-91066793dcc2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:58.294937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5d97235b-626b-4257-bdb8-91066793dcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:58.305406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:58.349610 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:58.350149 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602af90d-027c-4d6a-949f-48601483d9e7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:58.363120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 305/629] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:58.365031 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97235b-626b-4257-bdb8-91066793dcc2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:58.386697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff23150-5a73-4705-ab88-ee66074be82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:58.387373 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff23150-5a73-4705-ab88-ee66074be82d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.387654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff23150-5a73-4705-ab88-ee66074be82d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.387931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff23150-5a73-4705-ab88-ee66074be82d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.398424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff23150-5a73-4705-ab88-ee66074be82d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:58.399886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff23150-5a73-4705-ab88-ee66074be82d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1ff23150-5a73-4705-ab88-ee66074be82d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:58.415078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:58.416041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:58.416909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:58.416909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:58.416909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:58.416909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:58.418922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 325/630] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:58.419928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff23150-5a73-4705-ab88-ee66074be82d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:58.465196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97235b-626b-4257-bdb8-91066793dcc2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:58.499288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c87fa-4227-4910-92ad-62e1b05c5b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:58.499924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c87fa-4227-4910-92ad-62e1b05c5b51 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.500201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c87fa-4227-4910-92ad-62e1b05c5b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.500465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c87fa-4227-4910-92ad-62e1b05c5b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.511444 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c87fa-4227-4910-92ad-62e1b05c5b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:58.512931 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c87fa-4227-4910-92ad-62e1b05c5b51 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:58.519137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff23150-5a73-4705-ab88-ee66074be82d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-518c87fa-4227-4910-92ad-62e1b05c5b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:58.526794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:58.527804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:58.528680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:58.528680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:58.528680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:58.528680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:58.528680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 306/631] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:58.529257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12ab5a-23ec-4d16-936d-64a1b81a7db6 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.529351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12ab5a-23ec-4d16-936d-64a1b81a7db6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.529636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12ab5a-23ec-4d16-936d-64a1b81a7db6 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.531181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:58.534265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 307/632] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:46:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:46:58.535097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12ab5a-23ec-4d16-936d-64a1b81a7db6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:58.559368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15f161-f1a9-435e-9955-7ba2d064af11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:58.559993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15f161-f1a9-435e-9955-7ba2d064af11 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.560276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15f161-f1a9-435e-9955-7ba2d064af11 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.560554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15f161-f1a9-435e-9955-7ba2d064af11 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.561273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15f161-f1a9-435e-9955-7ba2d064af11 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:58.561579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15f161-f1a9-435e-9955-7ba2d064af11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:58.562978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15f161-f1a9-435e-9955-7ba2d064af11 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:58.571739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 326/633] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:58.572608 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15f161-f1a9-435e-9955-7ba2d064af11 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:58.581079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:58.640343 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12ab5a-23ec-4d16-936d-64a1b81a7db6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:58.672816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15f161-f1a9-435e-9955-7ba2d064af11 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:58.674337 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e261e-a17d-4dbd-bfb4-d72c019570d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:58.675020 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e261e-a17d-4dbd-bfb4-d72c019570d5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.675304 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e261e-a17d-4dbd-bfb4-d72c019570d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.675578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e261e-a17d-4dbd-bfb4-d72c019570d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.686442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e261e-a17d-4dbd-bfb4-d72c019570d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ed1e261e-a17d-4dbd-bfb4-d72c019570d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:58.691860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:58.693079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 308/634] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:58.693891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e261e-a17d-4dbd-bfb4-d72c019570d5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:58.711424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a85d3-7b27-4c08-9570-65559d6a1122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:58.712041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a85d3-7b27-4c08-9570-65559d6a1122 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.712322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a85d3-7b27-4c08-9570-65559d6a1122 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.712598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a85d3-7b27-4c08-9570-65559d6a1122 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.713286 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a85d3-7b27-4c08-9570-65559d6a1122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:58.713591 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a85d3-7b27-4c08-9570-65559d6a1122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:58.715000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a85d3-7b27-4c08-9570-65559d6a1122 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:58.727583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 327/635] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:58.728685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926d1bc-c4de-430a-a3d9-feb370f28f67 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.728973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926d1bc-c4de-430a-a3d9-feb370f28f67 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.729248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926d1bc-c4de-430a-a3d9-feb370f28f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.746893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2926d1bc-c4de-430a-a3d9-feb370f28f67 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:46:58.748079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2926d1bc-c4de-430a-a3d9-feb370f28f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:46:58.774118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:58.795355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:58.798899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e261e-a17d-4dbd-bfb4-d72c019570d5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:58.816001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 328/636] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:46:58.817036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926d1bc-c4de-430a-a3d9-feb370f28f67 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:58.836496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373d247-8b0a-4de6-be88-fe1256035982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:58.837118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373d247-8b0a-4de6-be88-fe1256035982 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.837407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373d247-8b0a-4de6-be88-fe1256035982 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.837686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373d247-8b0a-4de6-be88-fe1256035982 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.838459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373d247-8b0a-4de6-be88-fe1256035982 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:58.838769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373d247-8b0a-4de6-be88-fe1256035982 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:58.840175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373d247-8b0a-4de6-be88-fe1256035982 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:58.852709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 309/637] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:58.853508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373d247-8b0a-4de6-be88-fe1256035982 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:58.919979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926d1bc-c4de-430a-a3d9-feb370f28f67 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:58.956633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373d247-8b0a-4de6-be88-fe1256035982 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:58.957687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3fd29-4789-4a28-a28b-8d2b4f04684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:58.958515 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3fd29-4789-4a28-a28b-8d2b4f04684d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.958827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3fd29-4789-4a28-a28b-8d2b4f04684d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.959115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3fd29-4789-4a28-a28b-8d2b4f04684d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.969193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3fd29-4789-4a28-a28b-8d2b4f04684d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:58.970630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3fd29-4789-4a28-a28b-8d2b4f04684d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-bfc3fd29-4789-4a28-a28b-8d2b4f04684d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:58.985777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:58.986839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:58.987961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:58.987961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:58.987961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:58.987961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:58.987961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 329/638] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:58.987961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3fd29-4789-4a28-a28b-8d2b4f04684d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:58.995543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b20332-1b36-4af3-b52a-fad5466bf75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:58.996237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b20332-1b36-4af3-b52a-fad5466bf75c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:58.996521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b20332-1b36-4af3-b52a-fad5466bf75c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:58.996798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b20332-1b36-4af3-b52a-fad5466bf75c None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:58.997525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b20332-1b36-4af3-b52a-fad5466bf75c None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:58.997829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b20332-1b36-4af3-b52a-fad5466bf75c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:58.999250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b20332-1b36-4af3-b52a-fad5466bf75c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.003993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:59.008333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 310/639] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:59.009158 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b20332-1b36-4af3-b52a-fad5466bf75c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.033274 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:59.093428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3fd29-4789-4a28-a28b-8d2b4f04684d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.110794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b20332-1b36-4af3-b52a-fad5466bf75c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.127339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123da4c9-f6af-46e0-92a5-33161bded008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.127982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123da4c9-f6af-46e0-92a5-33161bded008 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.128267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123da4c9-f6af-46e0-92a5-33161bded008 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.128547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123da4c9-f6af-46e0-92a5-33161bded008 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.135388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123da4c9-f6af-46e0-92a5-33161bded008 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:59.136844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123da4c9-f6af-46e0-92a5-33161bded008 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.143607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdc5ba-2687-40a9-ae95-9a6730f78939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.144237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdc5ba-2687-40a9-ae95-9a6730f78939 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.144518 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdc5ba-2687-40a9-ae95-9a6730f78939 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.145329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdc5ba-2687-40a9-ae95-9a6730f78939 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-123da4c9-f6af-46e0-92a5-33161bded008 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:46:59.149334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:59.150354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:59.151318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:46:59.151318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:46:59.151318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:46:59.151318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:46:59.151318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:46:59.151318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:46:59.151318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:59.151318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:59.151318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 330/640] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:59.151318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123da4c9-f6af-46e0-92a5-33161bded008 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.157719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdc5ba-2687-40a9-ae95-9a6730f78939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8ccdc5ba-2687-40a9-ae95-9a6730f78939 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:46:59.164478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:59.165544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 311/641] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:59.166455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdc5ba-2687-40a9-ae95-9a6730f78939 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.229420 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:59.247247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:59.253979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123da4c9-f6af-46e0-92a5-33161bded008 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.266670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdc5ba-2687-40a9-ae95-9a6730f78939 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.288045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068459c-8246-4987-b444-97bfe2a4402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.288679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068459c-8246-4987-b444-97bfe2a4402d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.288963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068459c-8246-4987-b444-97bfe2a4402d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.289241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068459c-8246-4987-b444-97bfe2a4402d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.295780 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068459c-8246-4987-b444-97bfe2a4402d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:59.297232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068459c-8246-4987-b444-97bfe2a4402d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.300438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb953c48-5ad5-43bd-9591-fb5243809dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.301054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb953c48-5ad5-43bd-9591-fb5243809dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.301340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb953c48-5ad5-43bd-9591-fb5243809dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.301619 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb953c48-5ad5-43bd-9591-fb5243809dc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.302309 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb953c48-5ad5-43bd-9591-fb5243809dc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:59.302616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb953c48-5ad5-43bd-9591-fb5243809dc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:59.304040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb953c48-5ad5-43bd-9591-fb5243809dc8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-8068459c-8246-4987-b444-97bfe2a4402d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:46:59.308637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:59.309668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:59.310620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:46:59.310620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:46:59.310620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:46:59.310620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:46:59.310620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:46:59.310620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:46:59.310620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:46:59.310620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:59.310620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 331/642] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:59.310620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068459c-8246-4987-b444-97bfe2a4402d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.321095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 312/643] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:59.321095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb953c48-5ad5-43bd-9591-fb5243809dc8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.413261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068459c-8246-4987-b444-97bfe2a4402d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.418387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb953c48-5ad5-43bd-9591-fb5243809dc8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.446541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bb1b5-ada8-45b2-96a3-b696f0a4193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.447206 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bb1b5-ada8-45b2-96a3-b696f0a4193c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.447490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bb1b5-ada8-45b2-96a3-b696f0a4193c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.447776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bb1b5-ada8-45b2-96a3-b696f0a4193c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.449757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:59.452948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9bfc1-3579-4012-bf3b-89182cddacf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.453561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9bfc1-3579-4012-bf3b-89182cddacf4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.453848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9bfc1-3579-4012-bf3b-89182cddacf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.454127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9bfc1-3579-4012-bf3b-89182cddacf4 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.454909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9bfc1-3579-4012-bf3b-89182cddacf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:59.455225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9bfc1-3579-4012-bf3b-89182cddacf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:59.456657 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9bfc1-3579-4012-bf3b-89182cddacf4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.458927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139bb1b5-ada8-45b2-96a3-b696f0a4193c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:59.460383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139bb1b5-ada8-45b2-96a3-b696f0a4193c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.468074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 313/644] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:59.468874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9bfc1-3579-4012-bf3b-89182cddacf4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.472737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-139bb1b5-ada8-45b2-96a3-b696f0a4193c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:59.475720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:59.481465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:59.482561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:59.482561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:59.482561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:59.482561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:46:59.482561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 332/645] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:59.482561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bb1b5-ada8-45b2-96a3-b696f0a4193c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.572597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9bfc1-3579-4012-bf3b-89182cddacf4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.585030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bb1b5-ada8-45b2-96a3-b696f0a4193c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.605892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cb844-825e-49d5-b632-e2a0d4b3d081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.606506 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cb844-825e-49d5-b632-e2a0d4b3d081 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.606807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cb844-825e-49d5-b632-e2a0d4b3d081 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.607090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cb844-825e-49d5-b632-e2a0d4b3d081 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.607801 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cb844-825e-49d5-b632-e2a0d4b3d081 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:59.608105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cb844-825e-49d5-b632-e2a0d4b3d081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:59.609466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cb844-825e-49d5-b632-e2a0d4b3d081 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.617653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 314/646] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:59.622184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963771b9-5f6e-4120-abd4-3f1c4d5b9a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.622319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cb844-825e-49d5-b632-e2a0d4b3d081 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.622407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963771b9-5f6e-4120-abd4-3f1c4d5b9a81 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.622407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963771b9-5f6e-4120-abd4-3f1c4d5b9a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.622407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963771b9-5f6e-4120-abd4-3f1c4d5b9a81 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.622407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963771b9-5f6e-4120-abd4-3f1c4d5b9a81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:59.622407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963771b9-5f6e-4120-abd4-3f1c4d5b9a81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:59.622407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963771b9-5f6e-4120-abd4-3f1c4d5b9a81 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.634887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 333/647] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:59.635718 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963771b9-5f6e-4120-abd4-3f1c4d5b9a81 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.682541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:59.684196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:59.722645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cb844-825e-49d5-b632-e2a0d4b3d081 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.736101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963771b9-5f6e-4120-abd4-3f1c4d5b9a81 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.756820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787a83f-7afe-4c08-abcf-483adb1544a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.757461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a83f-7afe-4c08-abcf-483adb1544a3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.757744 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a83f-7afe-4c08-abcf-483adb1544a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.758016 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a83f-7afe-4c08-abcf-483adb1544a3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.766298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787a83f-7afe-4c08-abcf-483adb1544a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:59.767780 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787a83f-7afe-4c08-abcf-483adb1544a3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.770290 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1f3d9-14fc-4cc1-8cdb-4f8d78174ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.770952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1f3d9-14fc-4cc1-8cdb-4f8d78174ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.771238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1f3d9-14fc-4cc1-8cdb-4f8d78174ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.771517 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1f3d9-14fc-4cc1-8cdb-4f8d78174ed8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.772299 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1f3d9-14fc-4cc1-8cdb-4f8d78174ed8 None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=162ba16d643747c287bfe8a8e59df235) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:59.772610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1f3d9-14fc-4cc1-8cdb-4f8d78174ed8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:59.774018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1f3d9-14fc-4cc1-8cdb-4f8d78174ed8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-e787a83f-7afe-4c08-abcf-483adb1544a3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:46:59.778868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:59.791376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:59.792558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:59.792558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:59.792558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:46:59.792558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:46:59.792558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:46:59.792558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:46:59.792558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:46:59.792558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:46:59.792558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 03:46:59.792558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:59.793201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 334/648] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/role_assignments?user.id=162ba16d643747c287bfe8a8e59df235&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:59.793299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 315/649] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:59.793382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1f3d9-14fc-4cc1-8cdb-4f8d78174ed8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.793484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787a83f-7afe-4c08-abcf-483adb1544a3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.899069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1f3d9-14fc-4cc1-8cdb-4f8d78174ed8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.899720 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787a83f-7afe-4c08-abcf-483adb1544a3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:46:59.918414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:59.931678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:46:59.943753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc5128-ca62-4adf-9bfc-b486730cf85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.944538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc5128-ca62-4adf-9bfc-b486730cf85e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.944927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc5128-ca62-4adf-9bfc-b486730cf85e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.945307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc5128-ca62-4adf-9bfc-b486730cf85e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.951185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da598ef6-e2cb-4dda-95fe-2e542faf5ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:46:59.951810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da598ef6-e2cb-4dda-95fe-2e542faf5ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:46:59.952088 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da598ef6-e2cb-4dda-95fe-2e542faf5ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:46:59.952363 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da598ef6-e2cb-4dda-95fe-2e542faf5ff2 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:46:59.953067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da598ef6-e2cb-4dda-95fe-2e542faf5ff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:46:59.953896 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da598ef6-e2cb-4dda-95fe-2e542faf5ff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:59.955119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da598ef6-e2cb-4dda-95fe-2e542faf5ff2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.960204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc5128-ca62-4adf-9bfc-b486730cf85e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:46:59.961632 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc5128-ca62-4adf-9bfc-b486730cf85e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:46:59.966850 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 335/650] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:46:59.967652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da598ef6-e2cb-4dda-95fe-2e542faf5ff2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-68bc5128-ca62-4adf-9bfc-b486730cf85e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:46:59.978165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:46:59.979423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:46:59.980374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:46:59.980374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:46:59.980374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:46:59.980374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:46:59.980374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 316/651] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:46:59.980667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc5128-ca62-4adf-9bfc-b486730cf85e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:00.078122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da598ef6-e2cb-4dda-95fe-2e542faf5ff2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:00.086647 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc5128-ca62-4adf-9bfc-b486730cf85e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:00.115048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ab8fd-40af-487e-a812-8e13301828ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:00.115830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ab8fd-40af-487e-a812-8e13301828ff None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:00.116115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ab8fd-40af-487e-a812-8e13301828ff None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:00.116394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ab8fd-40af-487e-a812-8e13301828ff None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/19779569e08a42148d2933f644e3b9fa/roles/f5a9eb03e88a4e66bb9a8b6180009f72` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:00.120310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d15c46-5bd2-479a-a63b-1c6d0e454dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:00.120928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d15c46-5bd2-479a-a63b-1c6d0e454dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:00.121209 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d15c46-5bd2-479a-a63b-1c6d0e454dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:00.121481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d15c46-5bd2-479a-a63b-1c6d0e454dd5 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:00.122271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d15c46-5bd2-479a-a63b-1c6d0e454dd5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:00.122577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d15c46-5bd2-479a-a63b-1c6d0e454dd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:00.124052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d15c46-5bd2-479a-a63b-1c6d0e454dd5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:00.132669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ab8fd-40af-487e-a812-8e13301828ff None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=19779569e08a42148d2933f644e3b9fa, role_id=f5a9eb03e88a4e66bb9a8b6180009f72)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:00.134380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ab8fd-40af-487e-a812-8e13301828ff None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:00.142717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 317/652] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jul 28 03:46:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:00.142717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d15c46-5bd2-479a-a63b-1c6d0e454dd5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:00.152377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 336/653] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:46:59 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/19779569e08a42148d2933f644e3b9fa/roles/f5a9eb03e88a4e66bb9a8b6180009f72 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:47:00.153160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ab8fd-40af-487e-a812-8e13301828ff None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:00.182809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:00.205741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:00.295411 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ab8fd-40af-487e-a812-8e13301828ff None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:00.306032 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d15c46-5bd2-479a-a63b-1c6d0e454dd5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:00.337812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8edbb9-2fd7-476d-8b1e-dd91e2bab55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:00.338365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8edbb9-2fd7-476d-8b1e-dd91e2bab55a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:00.338650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8edbb9-2fd7-476d-8b1e-dd91e2bab55a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:00.338965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8edbb9-2fd7-476d-8b1e-dd91e2bab55a None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:00.339811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8edbb9-2fd7-476d-8b1e-dd91e2bab55a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:00.340076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8edbb9-2fd7-476d-8b1e-dd91e2bab55a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:00.341569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8edbb9-2fd7-476d-8b1e-dd91e2bab55a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:00.346068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617216b-6da7-4c61-bb65-46e432ab732c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:00.347010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617216b-6da7-4c61-bb65-46e432ab732c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:00.347491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617216b-6da7-4c61-bb65-46e432ab732c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:00.348991 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617216b-6da7-4c61-bb65-46e432ab732c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:00.355169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 337/654] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:00.356018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8edbb9-2fd7-476d-8b1e-dd91e2bab55a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:00.357458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617216b-6da7-4c61-bb65-46e432ab732c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:00.359243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617216b-6da7-4c61-bb65-46e432ab732c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7617216b-6da7-4c61-bb65-46e432ab732c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:00.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:00.376391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:00.377323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:00.377323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:00.377323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:00.377323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:00.377323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:00.377323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:00.377323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:00.377323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:00.377323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 03:47:00.377323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:00.380091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 318/655] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:47:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:00.381106 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617216b-6da7-4c61-bb65-46e432ab732c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:00.461108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:00.483298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:00.484274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8edbb9-2fd7-476d-8b1e-dd91e2bab55a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:00.492614 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617216b-6da7-4c61-bb65-46e432ab732c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:00.521830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e73f5d-81d6-4946-9f7a-1c9ec50dc41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:00.522508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e73f5d-81d6-4946-9f7a-1c9ec50dc41a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:00.522814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e73f5d-81d6-4946-9f7a-1c9ec50dc41a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:00.523198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e73f5d-81d6-4946-9f7a-1c9ec50dc41a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:00.530733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b101d9-5340-4084-a3f8-2263ed230b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:00.531670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b101d9-5340-4084-a3f8-2263ed230b60 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:00.532157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b101d9-5340-4084-a3f8-2263ed230b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:00.532733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b101d9-5340-4084-a3f8-2263ed230b60 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:00.541589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b101d9-5340-4084-a3f8-2263ed230b60 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:00.543446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b101d9-5340-4084-a3f8-2263ed230b60 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-16b101d9-5340-4084-a3f8-2263ed230b60 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:00.559665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:00.560622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:00.561567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:00.561567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:00.561567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:00.561567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:00.561567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:00.561567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:00.561567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:00.561567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:00.561567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 03:47:00.561567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:00.564344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 319/656] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:47:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:00.565364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b101d9-5340-4084-a3f8-2263ed230b60 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:00.606877 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e73f5d-81d6-4946-9f7a-1c9ec50dc41a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:00.608452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e73f5d-81d6-4946-9f7a-1c9ec50dc41a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:00.702049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:00.703984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b101d9-5340-4084-a3f8-2263ed230b60 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:00.731033 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 338/657] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:47:00.732226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e73f5d-81d6-4946-9f7a-1c9ec50dc41a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:00.750457 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1254b60-c0f0-4ca0-b610-6dfb47f0a673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:00.751105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1254b60-c0f0-4ca0-b610-6dfb47f0a673 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:00.751441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1254b60-c0f0-4ca0-b610-6dfb47f0a673 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:00.751717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1254b60-c0f0-4ca0-b610-6dfb47f0a673 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:00.752891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1254b60-c0f0-4ca0-b610-6dfb47f0a673 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:00.753248 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1254b60-c0f0-4ca0-b610-6dfb47f0a673 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:00.754759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1254b60-c0f0-4ca0-b610-6dfb47f0a673 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:00.778870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 320/658] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jul 28 03:47:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:00.780049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1254b60-c0f0-4ca0-b610-6dfb47f0a673 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:00.802340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:00.849613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e73f5d-81d6-4946-9f7a-1c9ec50dc41a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:00.890858 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60672f67-fad5-44f4-84e8-e4e2f43fff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:00.891840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60672f67-fad5-44f4-84e8-e4e2f43fff83 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:00.892326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60672f67-fad5-44f4-84e8-e4e2f43fff83 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:00.892793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60672f67-fad5-44f4-84e8-e4e2f43fff83 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:00.893794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60672f67-fad5-44f4-84e8-e4e2f43fff83 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:00.894298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60672f67-fad5-44f4-84e8-e4e2f43fff83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:00.896275 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60672f67-fad5-44f4-84e8-e4e2f43fff83 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:00.908861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1254b60-c0f0-4ca0-b610-6dfb47f0a673 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:00.916218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 339/659] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Jul 28 03:47:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:00.917375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60672f67-fad5-44f4-84e8-e4e2f43fff83 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:00.955906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e167c75-583a-4f2d-bf16-2a797ae2fdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:00.956538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e167c75-583a-4f2d-bf16-2a797ae2fdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:00.956821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e167c75-583a-4f2d-bf16-2a797ae2fdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:00.957176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e167c75-583a-4f2d-bf16-2a797ae2fdd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:00.968278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e167c75-583a-4f2d-bf16-2a797ae2fdd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:00.969736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e167c75-583a-4f2d-bf16-2a797ae2fdd4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:00.975930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7e167c75-583a-4f2d-bf16-2a797ae2fdd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:00.985785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:00.986957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:00.987869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:00.987869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:00.987869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:00.987869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:00.987869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 321/660] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:00.988143 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e167c75-583a-4f2d-bf16-2a797ae2fdd4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:01.058159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:01.067728 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60672f67-fad5-44f4-84e8-e4e2f43fff83 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:01.104116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebce50-8c48-4641-9605-c129ddb91760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:01.104733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebce50-8c48-4641-9605-c129ddb91760 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:01.105018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebce50-8c48-4641-9605-c129ddb91760 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:01.105293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebce50-8c48-4641-9605-c129ddb91760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:01.110068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e167c75-583a-4f2d-bf16-2a797ae2fdd4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:01.152726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4838a-ef62-4f2b-8c57-ad29d50f3054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:01.153266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4838a-ef62-4f2b-8c57-ad29d50f3054 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:01.153550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4838a-ef62-4f2b-8c57-ad29d50f3054 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:01.153823 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4838a-ef62-4f2b-8c57-ad29d50f3054 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:01.195038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebce50-8c48-4641-9605-c129ddb91760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:01.197226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebce50-8c48-4641-9605-c129ddb91760 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:01.229878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4838a-ef62-4f2b-8c57-ad29d50f3054 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:01.231402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4838a-ef62-4f2b-8c57-ad29d50f3054 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:01.250711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:01.307073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:01.338699 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 340/661] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:47:01.338699 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebce50-8c48-4641-9605-c129ddb91760 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:01.367626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 322/662] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:47:01.368957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4838a-ef62-4f2b-8c57-ad29d50f3054 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:01.446444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebce50-8c48-4641-9605-c129ddb91760 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:01.484192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2947d-d7b1-4bb0-8d4b-88b69835eb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:01.484953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2947d-d7b1-4bb0-8d4b-88b69835eb47 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:01.485334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2947d-d7b1-4bb0-8d4b-88b69835eb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:01.485727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2947d-d7b1-4bb0-8d4b-88b69835eb47 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:01.486604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2947d-d7b1-4bb0-8d4b-88b69835eb47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:01.487218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2947d-d7b1-4bb0-8d4b-88b69835eb47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:01.488670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2947d-d7b1-4bb0-8d4b-88b69835eb47 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:01.496548 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:01.498055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4838a-ef62-4f2b-8c57-ad29d50f3054 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:01.503343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 341/663] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:01.505363 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2947d-d7b1-4bb0-8d4b-88b69835eb47 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:01.545261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5f095-1b43-4ec6-9377-e1aaa1086ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:01.546218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5f095-1b43-4ec6-9377-e1aaa1086ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:01.546654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5f095-1b43-4ec6-9377-e1aaa1086ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:01.547368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5f095-1b43-4ec6-9377-e1aaa1086ba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:01.559822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5f095-1b43-4ec6-9377-e1aaa1086ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:01.562034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5f095-1b43-4ec6-9377-e1aaa1086ba4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-64b5f095-1b43-4ec6-9377-e1aaa1086ba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:01.583042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:01.584164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:01.585099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:01.585099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:01.585099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:01.585099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:01.590240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 323/664] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:01.590240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5f095-1b43-4ec6-9377-e1aaa1086ba4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:01.615407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:01.640212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2947d-d7b1-4bb0-8d4b-88b69835eb47 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:01.688553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a668-7990-451e-b8cf-d56e7ec543b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:01.689761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a668-7990-451e-b8cf-d56e7ec543b4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:01.690672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a668-7990-451e-b8cf-d56e7ec543b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:01.691296 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a668-7990-451e-b8cf-d56e7ec543b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:01.706764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2a668-7990-451e-b8cf-d56e7ec543b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:01.709153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2a668-7990-451e-b8cf-d56e7ec543b4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:01.710334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5f095-1b43-4ec6-9377-e1aaa1086ba4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-6ea2a668-7990-451e-b8cf-d56e7ec543b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:01.729050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:01.729963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:01.730851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:01.730851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:01.730851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:01.730851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:01.733780 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 342/665] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:01.734775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a668-7990-451e-b8cf-d56e7ec543b4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:01.751008 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dbee7-7edc-4fe2-80d2-e55f5398bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:01.752184 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dbee7-7edc-4fe2-80d2-e55f5398bc3a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:01.752661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dbee7-7edc-4fe2-80d2-e55f5398bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:01.753131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dbee7-7edc-4fe2-80d2-e55f5398bc3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:01.763358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dbee7-7edc-4fe2-80d2-e55f5398bc3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:01.765665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dbee7-7edc-4fe2-80d2-e55f5398bc3a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-786dbee7-7edc-4fe2-80d2-e55f5398bc3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:01.780905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:01.781845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:01.782742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:01.782742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:01.782742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:01.782742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:01.782742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:01.782742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:01.782742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:01.782742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:01.782742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 324/666] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:01.783264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dbee7-7edc-4fe2-80d2-e55f5398bc3a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:01.801380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:01.867597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:01.871884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a668-7990-451e-b8cf-d56e7ec543b4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:01.893702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dbee7-7edc-4fe2-80d2-e55f5398bc3a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:01.918759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc61777-9616-41ff-809c-1ccc3e01d919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:01.920335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc61777-9616-41ff-809c-1ccc3e01d919 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:01.920816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc61777-9616-41ff-809c-1ccc3e01d919 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:01.921509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc61777-9616-41ff-809c-1ccc3e01d919 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:01.922632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc61777-9616-41ff-809c-1ccc3e01d919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:01.923135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc61777-9616-41ff-809c-1ccc3e01d919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:01.925308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc61777-9616-41ff-809c-1ccc3e01d919 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:01.931988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c89b5-de17-4112-a39f-86c2e62e3276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:01.932689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c89b5-de17-4112-a39f-86c2e62e3276 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:01.933058 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c89b5-de17-4112-a39f-86c2e62e3276 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:01.933400 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c89b5-de17-4112-a39f-86c2e62e3276 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:01.934253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c89b5-de17-4112-a39f-86c2e62e3276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:01.934561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c89b5-de17-4112-a39f-86c2e62e3276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:01.936045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c89b5-de17-4112-a39f-86c2e62e3276 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:01.943805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 343/667] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:01.944854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc61777-9616-41ff-809c-1ccc3e01d919 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:01.950846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 325/668] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:01.952192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a1818-b39d-44db-a4c2-08f01c29cd63 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:01.952577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a1818-b39d-44db-a4c2-08f01c29cd63 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:01.952971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a1818-b39d-44db-a4c2-08f01c29cd63 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:01.954924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 326/669] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:01.955831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a1818-b39d-44db-a4c2-08f01c29cd63 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:02.054227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc61777-9616-41ff-809c-1ccc3e01d919 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:02.063912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a1818-b39d-44db-a4c2-08f01c29cd63 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:02.067425 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:02.093448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23555712-4372-4fda-99d6-b5a2d23ddeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:02.094068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23555712-4372-4fda-99d6-b5a2d23ddeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.094354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23555712-4372-4fda-99d6-b5a2d23ddeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.094630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23555712-4372-4fda-99d6-b5a2d23ddeb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.095412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23555712-4372-4fda-99d6-b5a2d23ddeb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:02.095722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23555712-4372-4fda-99d6-b5a2d23ddeb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:02.097154 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23555712-4372-4fda-99d6-b5a2d23ddeb0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:02.099775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5b801-bf78-483a-bbf4-88c32175b623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:02.100409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5b801-bf78-483a-bbf4-88c32175b623 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.100686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5b801-bf78-483a-bbf4-88c32175b623 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.100955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5b801-bf78-483a-bbf4-88c32175b623 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.103977 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:02.108353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5b801-bf78-483a-bbf4-88c32175b623 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:02.114838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 344/670] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:02.114990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5b801-bf78-483a-bbf4-88c32175b623 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:02.115076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23555712-4372-4fda-99d6-b5a2d23ddeb0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-44a5b801-bf78-483a-bbf4-88c32175b623 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:02.126930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:02.127850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:02.128709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:02.128709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:02.128709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:02.128709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:02.128709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:02.128709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:02.128709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:02.128709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:02.128709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 327/671] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:02.130142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6964a-10b0-4351-a09e-1a9ccb303ee0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.130142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6964a-10b0-4351-a09e-1a9ccb303ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.130317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6964a-10b0-4351-a09e-1a9ccb303ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.148678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-37e6964a-10b0-4351-a09e-1a9ccb303ee0 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:02.149972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-37e6964a-10b0-4351-a09e-1a9ccb303ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:02.219459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 328/672] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:47:02.220310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6964a-10b0-4351-a09e-1a9ccb303ee0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:02.232906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23555712-4372-4fda-99d6-b5a2d23ddeb0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:02.267412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c95f7-f883-47fe-abd7-3644ad3460d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:02.268050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c95f7-f883-47fe-abd7-3644ad3460d3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.268344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c95f7-f883-47fe-abd7-3644ad3460d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.268612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c95f7-f883-47fe-abd7-3644ad3460d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.277407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c95f7-f883-47fe-abd7-3644ad3460d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:02.278879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c95f7-f883-47fe-abd7-3644ad3460d3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-608c95f7-f883-47fe-abd7-3644ad3460d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:02.290691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:02.291600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:02.292513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:02.292513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:02.292513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:02.292513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:02.292513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:02.292513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:02.292513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:02.292513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:02.292513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 345/673] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:02.293254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c95f7-f883-47fe-abd7-3644ad3460d3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:02.323703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:02.329999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6964a-10b0-4351-a09e-1a9ccb303ee0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:02.356369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:02.365613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed1c97-7c8f-4578-ab27-c030b652802b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:02.366222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed1c97-7c8f-4578-ab27-c030b652802b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.366502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed1c97-7c8f-4578-ab27-c030b652802b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.366776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed1c97-7c8f-4578-ab27-c030b652802b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.367612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed1c97-7c8f-4578-ab27-c030b652802b None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=d8967641dd624dacb5ff12c8e22599bd) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:02.367921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed1c97-7c8f-4578-ab27-c030b652802b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:02.369361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed1c97-7c8f-4578-ab27-c030b652802b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:02.375132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 329/674] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:02 2022] GET /identity/v3/role_assignments?user.id=d8967641dd624dacb5ff12c8e22599bd&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:02.376356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7002-e299-49ff-a6d3-5c666b0f4a0b None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.376643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7002-e299-49ff-a6d3-5c666b0f4a0b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.376912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7002-e299-49ff-a6d3-5c666b0f4a0b None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.378096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 330/675] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:02.378804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db7002-e299-49ff-a6d3-5c666b0f4a0b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:02.405866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c95f7-f883-47fe-abd7-3644ad3460d3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:02.445883 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c5698-850f-4a92-959a-004c4f57f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:02.446516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c5698-850f-4a92-959a-004c4f57f905 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.446824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c5698-850f-4a92-959a-004c4f57f905 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.447104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c5698-850f-4a92-959a-004c4f57f905 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.447876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c5698-850f-4a92-959a-004c4f57f905 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:02.448189 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c5698-850f-4a92-959a-004c4f57f905 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:02.449648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c5698-850f-4a92-959a-004c4f57f905 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:02.470053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 346/676] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:02.470876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c5698-850f-4a92-959a-004c4f57f905 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:02.503044 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db7002-e299-49ff-a6d3-5c666b0f4a0b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:02.552846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f73e84-6d2e-4b4f-826b-b5d8c254e127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:02.552846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f73e84-6d2e-4b4f-826b-b5d8c254e127 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.552846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f73e84-6d2e-4b4f-826b-b5d8c254e127 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.552846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f73e84-6d2e-4b4f-826b-b5d8c254e127 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.564948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f73e84-6d2e-4b4f-826b-b5d8c254e127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b0f73e84-6d2e-4b4f-826b-b5d8c254e127 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:02.571012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:02.571930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:02.572761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 331/677] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:02.573456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56569f5c-5bd9-4f5b-9bee-cf211bcbb902 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.573916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56569f5c-5bd9-4f5b-9bee-cf211bcbb902 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.574027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56569f5c-5bd9-4f5b-9bee-cf211bcbb902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.579330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:02.590761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c5698-850f-4a92-959a-004c4f57f905 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:02.593961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-56569f5c-5bd9-4f5b-9bee-cf211bcbb902 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:02.595258 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-56569f5c-5bd9-4f5b-9bee-cf211bcbb902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:02.630478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d3bb6-e03c-49ad-9aa1-75f8e7348ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:02.631161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d3bb6-e03c-49ad-9aa1-75f8e7348ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.631448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d3bb6-e03c-49ad-9aa1-75f8e7348ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.631725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d3bb6-e03c-49ad-9aa1-75f8e7348ea8 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.632486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d3bb6-e03c-49ad-9aa1-75f8e7348ea8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:02.632799 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d3bb6-e03c-49ad-9aa1-75f8e7348ea8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:02.634253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d3bb6-e03c-49ad-9aa1-75f8e7348ea8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:02.646662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:02.650960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 347/678] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:02.650960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d3bb6-e03c-49ad-9aa1-75f8e7348ea8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:02.682818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 332/679] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:47:02.682818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56569f5c-5bd9-4f5b-9bee-cf211bcbb902 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:02.766827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d3bb6-e03c-49ad-9aa1-75f8e7348ea8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:02.801837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56569f5c-5bd9-4f5b-9bee-cf211bcbb902 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:02.804978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a708f-64a6-4a70-86d5-4fd64883eb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:02.805296 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a708f-64a6-4a70-86d5-4fd64883eb45 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.805580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a708f-64a6-4a70-86d5-4fd64883eb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.805853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a708f-64a6-4a70-86d5-4fd64883eb45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.806641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a708f-64a6-4a70-86d5-4fd64883eb45 None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=e2cf2558517445c79e47c3b67501ca09) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:02.806979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a708f-64a6-4a70-86d5-4fd64883eb45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:02.808409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a708f-64a6-4a70-86d5-4fd64883eb45 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:02.814480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 348/680] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:02 2022] GET /identity/v3/role_assignments?user.id=e2cf2558517445c79e47c3b67501ca09&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:02.815121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a708f-64a6-4a70-86d5-4fd64883eb45 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:02.834696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:02.853119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2653434-a448-48b2-8c3b-46a1e552d649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:02.854718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653434-a448-48b2-8c3b-46a1e552d649 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:02.855217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653434-a448-48b2-8c3b-46a1e552d649 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:02.855652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653434-a448-48b2-8c3b-46a1e552d649 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:02.856547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2653434-a448-48b2-8c3b-46a1e552d649 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:02.857016 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2653434-a448-48b2-8c3b-46a1e552d649 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:02.859693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2653434-a448-48b2-8c3b-46a1e552d649 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:02.871166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 333/681] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:47:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:02.872163 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2653434-a448-48b2-8c3b-46a1e552d649 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:02.927752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:02.967518 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a708f-64a6-4a70-86d5-4fd64883eb45 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:03.005512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2653434-a448-48b2-8c3b-46a1e552d649 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:03.014575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad93a2-f373-4b3a-870f-fd08b805c3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:03.015642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad93a2-f373-4b3a-870f-fd08b805c3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:03.016067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad93a2-f373-4b3a-870f-fd08b805c3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:03.016243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad93a2-f373-4b3a-870f-fd08b805c3d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:03.016996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad93a2-f373-4b3a-870f-fd08b805c3d4 None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=6e31408bfea4412d8f68aea9b3c7528d) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:03.017306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad93a2-f373-4b3a-870f-fd08b805c3d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:03.019062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad93a2-f373-4b3a-870f-fd08b805c3d4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:03.027376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 349/682] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:02 2022] GET /identity/v3/role_assignments?user.id=6e31408bfea4412d8f68aea9b3c7528d&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:03.027962 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad93a2-f373-4b3a-870f-fd08b805c3d4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:03.055697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e079c72-3c11-4de9-be32-2f0a4abe4b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:03.056777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e079c72-3c11-4de9-be32-2f0a4abe4b2d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:03.057130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e079c72-3c11-4de9-be32-2f0a4abe4b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:03.057481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e079c72-3c11-4de9-be32-2f0a4abe4b2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:03.071236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e079c72-3c11-4de9-be32-2f0a4abe4b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5e079c72-3c11-4de9-be32-2f0a4abe4b2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:03.076863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:03.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:03.078890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 334/683] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:47:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:03.085160 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e079c72-3c11-4de9-be32-2f0a4abe4b2d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:03.147128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:03.182333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad93a2-f373-4b3a-870f-fd08b805c3d4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:03.226352 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7256e7-3430-42f1-874e-9d1378212b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:03.227443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7256e7-3430-42f1-874e-9d1378212b76 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:03.228417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7256e7-3430-42f1-874e-9d1378212b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:03.228932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7256e7-3430-42f1-874e-9d1378212b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:03.233775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e079c72-3c11-4de9-be32-2f0a4abe4b2d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:03.235627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:03.245535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7256e7-3430-42f1-874e-9d1378212b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:03.248148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7256e7-3430-42f1-874e-9d1378212b76 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-6b7256e7-3430-42f1-874e-9d1378212b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:03.270255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:03.271263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:03.272159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:03.272159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:03.272159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:03.272159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:03.275198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 350/684] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:03.280866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7256e7-3430-42f1-874e-9d1378212b76 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:03.293125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d2346-51d5-4ee4-8f52-e387974c9940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:03.293662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d2346-51d5-4ee4-8f52-e387974c9940 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:03.293947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d2346-51d5-4ee4-8f52-e387974c9940 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:03.294232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d2346-51d5-4ee4-8f52-e387974c9940 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:03.294993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d2346-51d5-4ee4-8f52-e387974c9940 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:03.295303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d2346-51d5-4ee4-8f52-e387974c9940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:03.296734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d2346-51d5-4ee4-8f52-e387974c9940 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:03.306126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 335/685] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:47:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:03.311608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d2346-51d5-4ee4-8f52-e387974c9940 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:03.395358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7256e7-3430-42f1-874e-9d1378212b76 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:03.418153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d2346-51d5-4ee4-8f52-e387974c9940 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:03.427463 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:03.434109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bded723-7d5e-4e3b-808e-7a30c43d7009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:03.435097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bded723-7d5e-4e3b-808e-7a30c43d7009 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:03.435578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bded723-7d5e-4e3b-808e-7a30c43d7009 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:03.436024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bded723-7d5e-4e3b-808e-7a30c43d7009 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:03.436884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bded723-7d5e-4e3b-808e-7a30c43d7009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:03.439062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bded723-7d5e-4e3b-808e-7a30c43d7009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:03.439062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bded723-7d5e-4e3b-808e-7a30c43d7009 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:03.454058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 351/686] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:03.458244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661b84d-6aa1-4daf-88b3-6361f40704eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:03.458244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661b84d-6aa1-4daf-88b3-6361f40704eb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:03.458244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661b84d-6aa1-4daf-88b3-6361f40704eb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:03.458244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661b84d-6aa1-4daf-88b3-6361f40704eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:03.461235 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bded723-7d5e-4e3b-808e-7a30c43d7009 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:03.470773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661b84d-6aa1-4daf-88b3-6361f40704eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:03.472261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661b84d-6aa1-4daf-88b3-6361f40704eb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5661b84d-6aa1-4daf-88b3-6361f40704eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:03.488799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:03.489767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:03.490651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:03.490651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:03.490651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:03.490651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:03.490651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 336/687] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:03.495821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661b84d-6aa1-4daf-88b3-6361f40704eb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:03.515910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:03.574990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bded723-7d5e-4e3b-808e-7a30c43d7009 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:03.607462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661b84d-6aa1-4daf-88b3-6361f40704eb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:03.611740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb395f-7c79-4f6c-bd11-d77d0d818f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:03.612957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb395f-7c79-4f6c-bd11-d77d0d818f66 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:03.613991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb395f-7c79-4f6c-bd11-d77d0d818f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:03.614696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb395f-7c79-4f6c-bd11-d77d0d818f66 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:03.622974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb395f-7c79-4f6c-bd11-d77d0d818f66 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:03.624855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb395f-7c79-4f6c-bd11-d77d0d818f66 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-8fbb395f-7c79-4f6c-bd11-d77d0d818f66 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:03.638060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:03.639098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:03.640030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:03.640030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:03.640030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:03.640030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:03.640030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:03.640030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:03.640030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:03.640030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:03.640030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:03.640030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:03.643211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 352/688] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jul 28 03:47:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:03.650037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb395f-7c79-4f6c-bd11-d77d0d818f66 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:03.650207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166fcaf-60fc-4c0b-99fe-e2902f8a2a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:03.650898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166fcaf-60fc-4c0b-99fe-e2902f8a2a37 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:03.651181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166fcaf-60fc-4c0b-99fe-e2902f8a2a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:03.651459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166fcaf-60fc-4c0b-99fe-e2902f8a2a37 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:03.652180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166fcaf-60fc-4c0b-99fe-e2902f8a2a37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:03.652490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166fcaf-60fc-4c0b-99fe-e2902f8a2a37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:03.653908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166fcaf-60fc-4c0b-99fe-e2902f8a2a37 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:03.668154 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 337/689] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:03.673746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166fcaf-60fc-4c0b-99fe-e2902f8a2a37 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:03.706679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:03.760893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb395f-7c79-4f6c-bd11-d77d0d818f66 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:03.763165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:03.798666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c178f4-38e2-4007-b16d-f9b610ca643b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:03.799190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166fcaf-60fc-4c0b-99fe-e2902f8a2a37 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:03.799921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c178f4-38e2-4007-b16d-f9b610ca643b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:03.800406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c178f4-38e2-4007-b16d-f9b610ca643b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:03.801430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c178f4-38e2-4007-b16d-f9b610ca643b None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:03.802519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c178f4-38e2-4007-b16d-f9b610ca643b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:03.803083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c178f4-38e2-4007-b16d-f9b610ca643b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:03.804744 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c178f4-38e2-4007-b16d-f9b610ca643b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:03.824065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 353/690] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jul 28 03:47:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:03.829138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c178f4-38e2-4007-b16d-f9b610ca643b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:03.935669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c178f4-38e2-4007-b16d-f9b610ca643b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:03.946261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:03.951488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90495e-063f-4bce-a7ac-3a0ec6b5c724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:03.952152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90495e-063f-4bce-a7ac-3a0ec6b5c724 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:03.952437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90495e-063f-4bce-a7ac-3a0ec6b5c724 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:03.952713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90495e-063f-4bce-a7ac-3a0ec6b5c724 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:03.966827 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90495e-063f-4bce-a7ac-3a0ec6b5c724 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:03.968726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90495e-063f-4bce-a7ac-3a0ec6b5c724 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-0f90495e-063f-4bce-a7ac-3a0ec6b5c724 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:03.988729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:03.989907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:03.990957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:03.990957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:03.990957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:03.990957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:03.990957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:03.990957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:03.990957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:03.990957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:03.990957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:47:03.990957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:03.997745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fc3c3-64d7-4d78-b2df-9eed5a49bdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:03.999449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fc3c3-64d7-4d78-b2df-9eed5a49bdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:04.000537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fc3c3-64d7-4d78-b2df-9eed5a49bdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:04.000969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 338/691] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:04.001462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fc3c3-64d7-4d78-b2df-9eed5a49bdf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:04.006259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90495e-063f-4bce-a7ac-3a0ec6b5c724 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:04.106096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fc3c3-64d7-4d78-b2df-9eed5a49bdf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:04.107971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fc3c3-64d7-4d78-b2df-9eed5a49bdf6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:04.181271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90495e-063f-4bce-a7ac-3a0ec6b5c724 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:04.214940 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 354/692] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:47:04.220071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fc3c3-64d7-4d78-b2df-9eed5a49bdf6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:04.239280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:04.243497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a0749-17d1-4272-a055-867dbf5679d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:04.244136 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a0749-17d1-4272-a055-867dbf5679d1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:04.244481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a0749-17d1-4272-a055-867dbf5679d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:04.244748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a0749-17d1-4272-a055-867dbf5679d1 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:04.245604 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a0749-17d1-4272-a055-867dbf5679d1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:04.246109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a0749-17d1-4272-a055-867dbf5679d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:04.247401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a0749-17d1-4272-a055-867dbf5679d1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:04.256778 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:04.297280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 339/693] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jul 28 03:47:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:04.303047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a0749-17d1-4272-a055-867dbf5679d1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:04.357098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fc3c3-64d7-4d78-b2df-9eed5a49bdf6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:04.424534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a5850-b089-4cc2-b711-c8bc0e872aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:04.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a5850-b089-4cc2-b711-c8bc0e872aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:04.426295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a5850-b089-4cc2-b711-c8bc0e872aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:04.426864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a5850-b089-4cc2-b711-c8bc0e872aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:04.438810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a0749-17d1-4272-a055-867dbf5679d1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:04.447703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a5850-b089-4cc2-b711-c8bc0e872aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:04.449523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a5850-b089-4cc2-b711-c8bc0e872aa5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:04.468907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea8823-2c8e-4a01-8c47-9e03a7550990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:04.469484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea8823-2c8e-4a01-8c47-9e03a7550990 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:04.469723 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea8823-2c8e-4a01-8c47-9e03a7550990 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:04.469997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea8823-2c8e-4a01-8c47-9e03a7550990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-147a5850-b089-4cc2-b711-c8bc0e872aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:04.476413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:04.477269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:04.478152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:04.478152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:04.478152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:04.478152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:04.480393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 355/694] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:04.491866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a5850-b089-4cc2-b711-c8bc0e872aa5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:04.530770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:04.544982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea8823-2c8e-4a01-8c47-9e03a7550990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:04.546619 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea8823-2c8e-4a01-8c47-9e03a7550990 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:04.612498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:04.654504 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 340/695] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 03:47:04.659710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea8823-2c8e-4a01-8c47-9e03a7550990 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:04.703466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a5850-b089-4cc2-b711-c8bc0e872aa5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:04.754963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfadc8-544b-4f25-baa1-210d7f4106d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:04.755871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfadc8-544b-4f25-baa1-210d7f4106d9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:04.756312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfadc8-544b-4f25-baa1-210d7f4106d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:04.756726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfadc8-544b-4f25-baa1-210d7f4106d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:04.757600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfadc8-544b-4f25-baa1-210d7f4106d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:04.758093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfadc8-544b-4f25-baa1-210d7f4106d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:04.760033 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfadc8-544b-4f25-baa1-210d7f4106d9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:04.780760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 356/696] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:04.783965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94ff65-d9bf-48ea-82d0-f96b62bffa4c None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:04.784258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94ff65-d9bf-48ea-82d0-f96b62bffa4c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:04.784533 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94ff65-d9bf-48ea-82d0-f96b62bffa4c None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:04.786291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 357/697] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:47:04.787324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94ff65-d9bf-48ea-82d0-f96b62bffa4c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:04.822142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:04.829175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea8823-2c8e-4a01-8c47-9e03a7550990 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:04.865589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc536215-8c8e-4d96-8785-656fc66bb3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:04.866477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc536215-8c8e-4d96-8785-656fc66bb3dc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:04.866930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc536215-8c8e-4d96-8785-656fc66bb3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:04.867369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc536215-8c8e-4d96-8785-656fc66bb3dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:04.877752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc536215-8c8e-4d96-8785-656fc66bb3dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:04.879430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc536215-8c8e-4d96-8785-656fc66bb3dc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-fc536215-8c8e-4d96-8785-656fc66bb3dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:04.894121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:04.895880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:04.896762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:04.896762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:04.896762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:04.896762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:04.896762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 341/698] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:04.897570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f936907-1fad-4bab-bd50-857ca8610d2f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:04.897773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f936907-1fad-4bab-bd50-857ca8610d2f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:04.898046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f936907-1fad-4bab-bd50-857ca8610d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:04.909639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94ff65-d9bf-48ea-82d0-f96b62bffa4c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:04.916456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1f936907-1fad-4bab-bd50-857ca8610d2f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:04.917642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1f936907-1fad-4bab-bd50-857ca8610d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:04.932329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:04.948322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcf2c7-35e0-4ccf-837a-033cd7927f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:04.949311 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf2c7-35e0-4ccf-837a-033cd7927f3c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:04.949821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf2c7-35e0-4ccf-837a-033cd7927f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:04.950334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf2c7-35e0-4ccf-837a-033cd7927f3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:04.958218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcf2c7-35e0-4ccf-837a-033cd7927f3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:04.959868 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcf2c7-35e0-4ccf-837a-033cd7927f3c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-a2fcf2c7-35e0-4ccf-837a-033cd7927f3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:04.971698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:04.973274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:04.974166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:04.974166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:04.974166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:04.974166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:04.974166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:04.974166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:04.974166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:04.974166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:04.981073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 358/699] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:04.982102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcf2c7-35e0-4ccf-837a-033cd7927f3c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:04.987436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 342/700] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:47:04.988377 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f936907-1fad-4bab-bd50-857ca8610d2f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:05.067152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:05.089303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f936907-1fad-4bab-bd50-857ca8610d2f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:05.090899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcf2c7-35e0-4ccf-837a-033cd7927f3c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:05.124859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb55aa-f7a8-4833-945d-bbef2da05176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:05.125480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb55aa-f7a8-4833-945d-bbef2da05176 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.125758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb55aa-f7a8-4833-945d-bbef2da05176 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.126027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb55aa-f7a8-4833-945d-bbef2da05176 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.126770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb55aa-f7a8-4833-945d-bbef2da05176 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:05.127123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb55aa-f7a8-4833-945d-bbef2da05176 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:05.128446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a6e5-03cb-4e42-bb8b-f1237304f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:05.129471 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb55aa-f7a8-4833-945d-bbef2da05176 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:05.130139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a6e5-03cb-4e42-bb8b-f1237304f00a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.130606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a6e5-03cb-4e42-bb8b-f1237304f00a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.131083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a6e5-03cb-4e42-bb8b-f1237304f00a None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.131956 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8a6e5-03cb-4e42-bb8b-f1237304f00a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:05.132440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8a6e5-03cb-4e42-bb8b-f1237304f00a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:05.133976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8a6e5-03cb-4e42-bb8b-f1237304f00a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:05.146416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 343/701] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:05.146416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb55aa-f7a8-4833-945d-bbef2da05176 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:05.157603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 359/702] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:05.158749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a6e5-03cb-4e42-bb8b-f1237304f00a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:05.204322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:05.243586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb55aa-f7a8-4833-945d-bbef2da05176 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:05.268144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a6e5-03cb-4e42-bb8b-f1237304f00a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:05.278326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7359747-92ee-4f90-b99f-d96f4ae0f921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:05.279160 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7359747-92ee-4f90-b99f-d96f4ae0f921 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.279276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7359747-92ee-4f90-b99f-d96f4ae0f921 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.279555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7359747-92ee-4f90-b99f-d96f4ae0f921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.291425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7359747-92ee-4f90-b99f-d96f4ae0f921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:05.292865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7359747-92ee-4f90-b99f-d96f4ae0f921 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:05.307249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae24a6-9d81-4fad-bd2d-cc8d2fdc6cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b7359747-92ee-4f90-b99f-d96f4ae0f921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:05.307479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:05.308336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:05.309273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:05.309273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:05.309273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:05.309273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:05.310630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae24a6-9d81-4fad-bd2d-cc8d2fdc6cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.311096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:05.311295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae24a6-9d81-4fad-bd2d-cc8d2fdc6cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.311778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae24a6-9d81-4fad-bd2d-cc8d2fdc6cd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.312758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae24a6-9d81-4fad-bd2d-cc8d2fdc6cd5 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=162ba16d643747c287bfe8a8e59df235) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:05.313247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae24a6-9d81-4fad-bd2d-cc8d2fdc6cd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:05.313396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 344/703] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:05.314223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7359747-92ee-4f90-b99f-d96f4ae0f921 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:05.315349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae24a6-9d81-4fad-bd2d-cc8d2fdc6cd5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:05.326979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 360/704] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:05 2022] GET /identity/v3/role_assignments?user.id=162ba16d643747c287bfe8a8e59df235&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=a37ae24c516b48c69e13721c013134a2 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:05.328186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae24a6-9d81-4fad-bd2d-cc8d2fdc6cd5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:05.426715 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7359747-92ee-4f90-b99f-d96f4ae0f921 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:05.451948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae24a6-9d81-4fad-bd2d-cc8d2fdc6cd5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:05.454985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:05.463735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f280f-464d-4604-bf08-0003530a989e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:05.464442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f280f-464d-4604-bf08-0003530a989e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.464680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f280f-464d-4604-bf08-0003530a989e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.464954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f280f-464d-4604-bf08-0003530a989e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.472499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f280f-464d-4604-bf08-0003530a989e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:05.474552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f280f-464d-4604-bf08-0003530a989e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-354f280f-464d-4604-bf08-0003530a989e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:05.486833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:05.487805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:05.488833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:05.488833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:05.488833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:05.488833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:05.488833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:05.488833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:05.488833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:05.488833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:05.492961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 345/705] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:05.494443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d87532-554f-4750-a34a-fbfe7bb440eb None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.494660 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d87532-554f-4750-a34a-fbfe7bb440eb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.494964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d87532-554f-4750-a34a-fbfe7bb440eb None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.496979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 346/706] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:05.497766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d87532-554f-4750-a34a-fbfe7bb440eb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:05.500933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c15900-c0e9-4842-b073-adb0e7b3e395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:05.501833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c15900-c0e9-4842-b073-adb0e7b3e395 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.502345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c15900-c0e9-4842-b073-adb0e7b3e395 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.502851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c15900-c0e9-4842-b073-adb0e7b3e395 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.504285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c15900-c0e9-4842-b073-adb0e7b3e395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:05.504798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c15900-c0e9-4842-b073-adb0e7b3e395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:05.506393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c15900-c0e9-4842-b073-adb0e7b3e395 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:05.519509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 361/707] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:05.521497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8fb5c-c38e-49db-b0e8-3b5d4a59e789 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.521995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8fb5c-c38e-49db-b0e8-3b5d4a59e789 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.522476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8fb5c-c38e-49db-b0e8-3b5d4a59e789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.542124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d6c8fb5c-c38e-49db-b0e8-3b5d4a59e789 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:05.543685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d6c8fb5c-c38e-49db-b0e8-3b5d4a59e789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:05.579325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:05.623989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d87532-554f-4750-a34a-fbfe7bb440eb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:05.626020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 362/708] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:47:05.627726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8fb5c-c38e-49db-b0e8-3b5d4a59e789 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:05.661886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ba48a-c835-4455-b94a-cb25985a2650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:05.662513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ba48a-c835-4455-b94a-cb25985a2650 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.662822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ba48a-c835-4455-b94a-cb25985a2650 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.663097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ba48a-c835-4455-b94a-cb25985a2650 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.664105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ba48a-c835-4455-b94a-cb25985a2650 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:05.664422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ba48a-c835-4455-b94a-cb25985a2650 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:05.665864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ba48a-c835-4455-b94a-cb25985a2650 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:05.682592 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 347/709] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:05.683695 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ba48a-c835-4455-b94a-cb25985a2650 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:05.739156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:05.746774 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8fb5c-c38e-49db-b0e8-3b5d4a59e789 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:05.784258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65358a0b-828e-4185-b7db-a54f3d61a232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:05.784900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65358a0b-828e-4185-b7db-a54f3d61a232 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.785796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65358a0b-828e-4185-b7db-a54f3d61a232 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.786023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65358a0b-828e-4185-b7db-a54f3d61a232 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.786982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65358a0b-828e-4185-b7db-a54f3d61a232 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:05.788414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65358a0b-828e-4185-b7db-a54f3d61a232 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:05.789472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ba48a-c835-4455-b94a-cb25985a2650 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:05.800460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:05.810653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 363/710] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:47:05 2022] POST /identity/v3/users => generated 254 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:05.812118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a116a7f-32ff-4203-beaa-7d349eba2db9 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.812398 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a116a7f-32ff-4203-beaa-7d349eba2db9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.812672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a116a7f-32ff-4203-beaa-7d349eba2db9 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.818273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 364/711] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:05.818273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a116a7f-32ff-4203-beaa-7d349eba2db9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:05.846211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78cf07-1c21-422b-bb69-e349ff91e527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:05.847069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78cf07-1c21-422b-bb69-e349ff91e527 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.847355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78cf07-1c21-422b-bb69-e349ff91e527 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.848191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78cf07-1c21-422b-bb69-e349ff91e527 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.849419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78cf07-1c21-422b-bb69-e349ff91e527 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:05.850897 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78cf07-1c21-422b-bb69-e349ff91e527 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5c78cf07-1c21-422b-bb69-e349ff91e527 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:05.862749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:05.863866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 03:47:05.864677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:05.865465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 348/712] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Jul 28 03:47:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:05.867309 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8626b-8fc7-4d8d-a314-198966e0b4e2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:05.867595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8626b-8fc7-4d8d-a314-198966e0b4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:05.867869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8626b-8fc7-4d8d-a314-198966e0b4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:05.888396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b4b8626b-8fc7-4d8d-a314-198966e0b4e2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:05.889709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b4b8626b-8fc7-4d8d-a314-198966e0b4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:05.958017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a116a7f-32ff-4203-beaa-7d349eba2db9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:05.988036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 349/713] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:47:05.990531 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8626b-8fc7-4d8d-a314-198966e0b4e2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:06.000204 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253c2f7-002b-417a-bcf9-d0bd8b90e6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:06.001077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253c2f7-002b-417a-bcf9-d0bd8b90e6ee None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:06.001561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253c2f7-002b-417a-bcf9-d0bd8b90e6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:06.002054 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253c2f7-002b-417a-bcf9-d0bd8b90e6ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:06.003109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253c2f7-002b-417a-bcf9-d0bd8b90e6ee None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=19779569e08a42148d2933f644e3b9fa) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:06.003630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253c2f7-002b-417a-bcf9-d0bd8b90e6ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:06.005285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253c2f7-002b-417a-bcf9-d0bd8b90e6ee None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:06.012619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 365/714] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:05 2022] GET /identity/v3/role_assignments?user.id=19779569e08a42148d2933f644e3b9fa&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:06.013704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253c2f7-002b-417a-bcf9-d0bd8b90e6ee None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:06.032529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:06.143141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8626b-8fc7-4d8d-a314-198966e0b4e2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:06.144481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253c2f7-002b-417a-bcf9-d0bd8b90e6ee None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:06.145226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:06.189318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448ffba-c181-4b4c-b53c-0eda381b5798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:06.189948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448ffba-c181-4b4c-b53c-0eda381b5798 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:06.190231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448ffba-c181-4b4c-b53c-0eda381b5798 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:06.190606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448ffba-c181-4b4c-b53c-0eda381b5798 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:06.192996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448ffba-c181-4b4c-b53c-0eda381b5798 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:06.192996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448ffba-c181-4b4c-b53c-0eda381b5798 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:06.194949 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448ffba-c181-4b4c-b53c-0eda381b5798 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:06.195424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adde13f-85b2-4ead-87bd-57e879a56640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:06.196733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adde13f-85b2-4ead-87bd-57e879a56640 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:06.197040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adde13f-85b2-4ead-87bd-57e879a56640 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:06.197315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adde13f-85b2-4ead-87bd-57e879a56640 None admin] PATH_INFO: `/v3/services/network` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:06.198239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adde13f-85b2-4ead-87bd-57e879a56640 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:06.200142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 350/715] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jul 28 03:47:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:06.200268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adde13f-85b2-4ead-87bd-57e879a56640 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:06.206258 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448ffba-c181-4b4c-b53c-0eda381b5798 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1adde13f-85b2-4ead-87bd-57e879a56640 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:06.214688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:06.217184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:06.218649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 366/716] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:06 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:06.225981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adde13f-85b2-4ead-87bd-57e879a56640 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:06.331877 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:06.338483 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448ffba-c181-4b4c-b53c-0eda381b5798 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:06.363446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adde13f-85b2-4ead-87bd-57e879a56640 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:06.378963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665b06-ec69-4e94-8c42-ace764b8ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:06.379669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665b06-ec69-4e94-8c42-ace764b8ea1b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:06.380034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665b06-ec69-4e94-8c42-ace764b8ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:06.380364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665b06-ec69-4e94-8c42-ace764b8ea1b None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:06.381122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29665b06-ec69-4e94-8c42-ace764b8ea1b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:06.381495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29665b06-ec69-4e94-8c42-ace764b8ea1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:06.382965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29665b06-ec69-4e94-8c42-ace764b8ea1b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:06.388987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 351/717] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jul 28 03:47:06 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:06.406096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0072c-0b39-43e1-97a6-10594f08f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:06.407010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0072c-0b39-43e1-97a6-10594f08f821 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:06.407482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0072c-0b39-43e1-97a6-10594f08f821 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:06.407900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0072c-0b39-43e1-97a6-10594f08f821 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:06.408795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0072c-0b39-43e1-97a6-10594f08f821 None admin] RBAC: Adding query filter params (name=network) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:06.409267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0072c-0b39-43e1-97a6-10594f08f821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:06.411856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0072c-0b39-43e1-97a6-10594f08f821 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:06.421835 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 367/718] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:06.422911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665b06-ec69-4e94-8c42-ace764b8ea1b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:06.459251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:06.538242 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665b06-ec69-4e94-8c42-ace764b8ea1b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:06.576764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c60e1-a3c3-41a5-884c-d6bcde08d37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:06.577411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c60e1-a3c3-41a5-884c-d6bcde08d37e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:06.577748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c60e1-a3c3-41a5-884c-d6bcde08d37e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:06.578029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c60e1-a3c3-41a5-884c-d6bcde08d37e None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:06.578738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c60e1-a3c3-41a5-884c-d6bcde08d37e None admin] RBAC: Adding query filter params (type=network) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:06.579083 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c60e1-a3c3-41a5-884c-d6bcde08d37e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:06.580637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c60e1-a3c3-41a5-884c-d6bcde08d37e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:06.587346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 352/719] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:07.713634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb4db9-cc0b-421c-b987-1036d60d1ca5 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:07.714144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb4db9-cc0b-421c-b987-1036d60d1ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:07.714247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb4db9-cc0b-421c-b987-1036d60d1ca5 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:07.715761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 368/720] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:07.731484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4ca91-b4a5-42a9-b3b9-be60b4d88928 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:07.731908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4ca91-b4a5-42a9-b3b9-be60b4d88928 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:07.732078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4ca91-b4a5-42a9-b3b9-be60b4d88928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:07.755730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-78a4ca91-b4a5-42a9-b3b9-be60b4d88928 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:07.756919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-78a4ca91-b4a5-42a9-b3b9-be60b4d88928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:07.841837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 353/721] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:47:07.875954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb4db9-cc0b-421c-b987-1036d60d1ca5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:08.007538 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb4db9-cc0b-421c-b987-1036d60d1ca5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:08.065147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7ec3a-81a3-4814-b916-58cb891c46a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:08.066406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7ec3a-81a3-4814-b916-58cb891c46a7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:08.066551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7ec3a-81a3-4814-b916-58cb891c46a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:08.066814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7ec3a-81a3-4814-b916-58cb891c46a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:08.095921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7ec3a-81a3-4814-b916-58cb891c46a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-10e7ec3a-81a3-4814-b916-58cb891c46a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:08.101416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:08.102334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:08.103203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 369/722] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:08.110089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4ca91-b4a5-42a9-b3b9-be60b4d88928 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:08.190199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:08.380070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4ca91-b4a5-42a9-b3b9-be60b4d88928 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:08.458610 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53924e-0ca5-4b93-80fe-08b069dcd371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:08.459292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53924e-0ca5-4b93-80fe-08b069dcd371 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:08.459580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53924e-0ca5-4b93-80fe-08b069dcd371 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:08.459859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53924e-0ca5-4b93-80fe-08b069dcd371 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:08.460595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53924e-0ca5-4b93-80fe-08b069dcd371 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:08.460906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53924e-0ca5-4b93-80fe-08b069dcd371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:08.462301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53924e-0ca5-4b93-80fe-08b069dcd371 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:08.485365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 354/723] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:47:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:08.495182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7ec3a-81a3-4814-b916-58cb891c46a7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:08.495994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:08.705422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7ec3a-81a3-4814-b916-58cb891c46a7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:08.747856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5edb24-f7ff-421d-8e24-1db466bb0800 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:08.748410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5edb24-f7ff-421d-8e24-1db466bb0800 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:08.748410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5edb24-f7ff-421d-8e24-1db466bb0800 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:08.749755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 355/724] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:47:08.751543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a76ec0-9bfd-4040-98d0-e755b250b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:08.752203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a76ec0-9bfd-4040-98d0-e755b250b37a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:08.752487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a76ec0-9bfd-4040-98d0-e755b250b37a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:08.752764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a76ec0-9bfd-4040-98d0-e755b250b37a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:08.757870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5972c4-65df-4de5-9a27-627c98d55eed None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:08.757870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5972c4-65df-4de5-9a27-627c98d55eed None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:08.757870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5972c4-65df-4de5-9a27-627c98d55eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:08.781998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a76ec0-9bfd-4040-98d0-e755b250b37a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:08.783519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a76ec0-9bfd-4040-98d0-e755b250b37a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:08.807644 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3c5972c4-65df-4de5-9a27-627c98d55eed None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:08.808182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3c5972c4-65df-4de5-9a27-627c98d55eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e4a76ec0-9bfd-4040-98d0-e755b250b37a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:08.825584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:08.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:08.830181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:08.830181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:08.830181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:08.830181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:08.835829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 370/725] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:08.837217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d94196-1d69-43c1-9723-d0fe31b5977a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:08.837529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d94196-1d69-43c1-9723-d0fe31b5977a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:08.837782 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d94196-1d69-43c1-9723-d0fe31b5977a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:08.845651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 371/726] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:08.845651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d94196-1d69-43c1-9723-d0fe31b5977a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:08.939236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:08.945798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 356/727] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:47:08.949226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efcf48-4d4c-4765-9617-5ea49b2473be None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:08.949752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efcf48-4d4c-4765-9617-5ea49b2473be None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:08.950222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efcf48-4d4c-4765-9617-5ea49b2473be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:08.990001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d94196-1d69-43c1-9723-d0fe31b5977a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:08.992089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-16efcf48-4d4c-4765-9617-5ea49b2473be None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:08.993528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-16efcf48-4d4c-4765-9617-5ea49b2473be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:09.010543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:09.026344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1e4ff-27cf-4b51-9b2d-16c6fe5472f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:09.027011 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1e4ff-27cf-4b51-9b2d-16c6fe5472f9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.027365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1e4ff-27cf-4b51-9b2d-16c6fe5472f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.027574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1e4ff-27cf-4b51-9b2d-16c6fe5472f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.028276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1e4ff-27cf-4b51-9b2d-16c6fe5472f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:09.028584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1e4ff-27cf-4b51-9b2d-16c6fe5472f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:09.029960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1e4ff-27cf-4b51-9b2d-16c6fe5472f9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:09.044739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 372/728] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:09.045288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1e4ff-27cf-4b51-9b2d-16c6fe5472f9 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:09.086808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 357/729] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:47:09.088319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efcf48-4d4c-4765-9617-5ea49b2473be None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:09.155891 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1e4ff-27cf-4b51-9b2d-16c6fe5472f9 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:09.170917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:09.198771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51c683-2998-4028-966f-4d140a9999bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:09.199663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51c683-2998-4028-966f-4d140a9999bb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.200253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51c683-2998-4028-966f-4d140a9999bb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.200536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51c683-2998-4028-966f-4d140a9999bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.214917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51c683-2998-4028-966f-4d140a9999bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:09.215890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efcf48-4d4c-4765-9617-5ea49b2473be None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-4e51c683-2998-4028-966f-4d140a9999bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:09.220826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:09.221778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:09.222680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 373/730] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:47:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:09.223925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51c683-2998-4028-966f-4d140a9999bb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:09.259906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5db4f7-36f4-4b77-96b2-ff3f5256a692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:09.260482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5db4f7-36f4-4b77-96b2-ff3f5256a692 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.260767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5db4f7-36f4-4b77-96b2-ff3f5256a692 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.261040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5db4f7-36f4-4b77-96b2-ff3f5256a692 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.271071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5db4f7-36f4-4b77-96b2-ff3f5256a692 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:09.273139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5db4f7-36f4-4b77-96b2-ff3f5256a692 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ee5db4f7-36f4-4b77-96b2-ff3f5256a692 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:09.286105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:09.292724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 358/731] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jul 28 03:47:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a0847-aaaa-4db0-85d4-ff07623599a5 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.293732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a0847-aaaa-4db0-85d4-ff07623599a5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.294456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a0847-aaaa-4db0-85d4-ff07623599a5 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.294456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 359/732] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:09.294456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a0847-aaaa-4db0-85d4-ff07623599a5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:09.303854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:09.340738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51c683-2998-4028-966f-4d140a9999bb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:09.378148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd5579-829a-4357-bbe7-856f39fa240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:09.379463 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd5579-829a-4357-bbe7-856f39fa240e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.379766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd5579-829a-4357-bbe7-856f39fa240e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.380018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd5579-829a-4357-bbe7-856f39fa240e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.391654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd5579-829a-4357-bbe7-856f39fa240e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d0bd5579-829a-4357-bbe7-856f39fa240e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:09.399019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:09.400273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 374/733] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:09.401148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd5579-829a-4357-bbe7-856f39fa240e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:09.404064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a0847-aaaa-4db0-85d4-ff07623599a5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:09.438913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:09.439612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0496a6-0d8b-4a17-b350-c15e2dc21fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:09.439978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0496a6-0d8b-4a17-b350-c15e2dc21fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.440210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0496a6-0d8b-4a17-b350-c15e2dc21fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.440517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0496a6-0d8b-4a17-b350-c15e2dc21fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.441369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0496a6-0d8b-4a17-b350-c15e2dc21fc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:09.441663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0496a6-0d8b-4a17-b350-c15e2dc21fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:09.443063 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0496a6-0d8b-4a17-b350-c15e2dc21fc5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:09.451325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 360/734] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:47:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:09.452502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7d15a-9a9f-47a6-b2e7-7d0bbe134392 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.452786 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7d15a-9a9f-47a6-b2e7-7d0bbe134392 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.453062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7d15a-9a9f-47a6-b2e7-7d0bbe134392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.470864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-80a7d15a-9a9f-47a6-b2e7-7d0bbe134392 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:09.472344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-80a7d15a-9a9f-47a6-b2e7-7d0bbe134392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:09.509678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd5579-829a-4357-bbe7-856f39fa240e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:09.534061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:09.540479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 361/735] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:47:09.541707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844c4a4-5f03-4dd1-be8c-19a0f49a3ec4 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.541989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844c4a4-5f03-4dd1-be8c-19a0f49a3ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.542260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844c4a4-5f03-4dd1-be8c-19a0f49a3ec4 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.543505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 362/736] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:09.544189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844c4a4-5f03-4dd1-be8c-19a0f49a3ec4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:09.551052 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5009ae6-bdce-47b9-967d-1aa2d9cc1fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:09.552090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5009ae6-bdce-47b9-967d-1aa2d9cc1fcf None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.552381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5009ae6-bdce-47b9-967d-1aa2d9cc1fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.552661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5009ae6-bdce-47b9-967d-1aa2d9cc1fcf None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.553473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5009ae6-bdce-47b9-967d-1aa2d9cc1fcf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:09.553784 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5009ae6-bdce-47b9-967d-1aa2d9cc1fcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:09.555257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5009ae6-bdce-47b9-967d-1aa2d9cc1fcf None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:09.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 375/737] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jul 28 03:47:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:09.574769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5009ae6-bdce-47b9-967d-1aa2d9cc1fcf None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:09.671576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:09.681362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5009ae6-bdce-47b9-967d-1aa2d9cc1fcf None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:09.716346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcf9f0-e546-4be4-bdbe-3f48893be32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:09.716981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcf9f0-e546-4be4-bdbe-3f48893be32d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.717260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcf9f0-e546-4be4-bdbe-3f48893be32d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.717535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcf9f0-e546-4be4-bdbe-3f48893be32d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.728086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcf9f0-e546-4be4-bdbe-3f48893be32d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:09.729542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcf9f0-e546-4be4-bdbe-3f48893be32d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-c8bcf9f0-e546-4be4-bdbe-3f48893be32d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:09.745045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:09.746022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:09.746933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:09.746933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:09.746933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:09.746933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:09.746933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 376/738] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:09.747933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1940980-081a-499a-a5be-69f7a405236d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.747933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1940980-081a-499a-a5be-69f7a405236d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.747933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1940980-081a-499a-a5be-69f7a405236d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.774722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844c4a4-5f03-4dd1-be8c-19a0f49a3ec4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:09.812289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526cebe-934b-441d-adb4-4f1840445e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:09.812924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526cebe-934b-441d-adb4-4f1840445e8a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.813205 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526cebe-934b-441d-adb4-4f1840445e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.813479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526cebe-934b-441d-adb4-4f1840445e8a None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.814221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526cebe-934b-441d-adb4-4f1840445e8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:09.814526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526cebe-934b-441d-adb4-4f1840445e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:09.815961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526cebe-934b-441d-adb4-4f1840445e8a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:09.825123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 363/739] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:47:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:09.825972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526cebe-934b-441d-adb4-4f1840445e8a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:09.881872 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d1940980-081a-499a-a5be-69f7a405236d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:09.883086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d1940980-081a-499a-a5be-69f7a405236d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:09.889693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:09.930294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526cebe-934b-441d-adb4-4f1840445e8a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:09.951331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 377/740] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 03:47:09.952093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1940980-081a-499a-a5be-69f7a405236d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:09.963906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba9f4e-f5da-452d-9e78-70ebdb1f7554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:09.964522 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba9f4e-f5da-452d-9e78-70ebdb1f7554 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:09.964807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba9f4e-f5da-452d-9e78-70ebdb1f7554 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:09.965078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba9f4e-f5da-452d-9e78-70ebdb1f7554 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:09.965762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba9f4e-f5da-452d-9e78-70ebdb1f7554 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:09.967148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba9f4e-f5da-452d-9e78-70ebdb1f7554 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:09.985226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 364/741] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:47:09 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:09.986242 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba9f4e-f5da-452d-9e78-70ebdb1f7554 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:10.062577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:10.096290 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1940980-081a-499a-a5be-69f7a405236d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:10.116173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba9f4e-f5da-452d-9e78-70ebdb1f7554 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:10.135233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:10.135917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e88644-b98f-4c56-9f18-b7ff94ea7235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:10.136567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e88644-b98f-4c56-9f18-b7ff94ea7235 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:10.136849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e88644-b98f-4c56-9f18-b7ff94ea7235 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:10.137118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e88644-b98f-4c56-9f18-b7ff94ea7235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:10.150387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e88644-b98f-4c56-9f18-b7ff94ea7235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:10.151892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e88644-b98f-4c56-9f18-b7ff94ea7235 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:10.156969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffbc05-7173-4b5a-9306-5e2c60885da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:10.157603 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffbc05-7173-4b5a-9306-5e2c60885da4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:10.157880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffbc05-7173-4b5a-9306-5e2c60885da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:10.158156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffbc05-7173-4b5a-9306-5e2c60885da4 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:10.158941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffbc05-7173-4b5a-9306-5e2c60885da4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:10.160563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffbc05-7173-4b5a-9306-5e2c60885da4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:10.161218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffbc05-7173-4b5a-9306-5e2c60885da4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-38e88644-b98f-4c56-9f18-b7ff94ea7235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:10.169736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:10.170657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:10.171788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:10.171788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:10.171788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:10.171788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:10.171788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 378/742] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:10.172106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e88644-b98f-4c56-9f18-b7ff94ea7235 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:10.178151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 365/743] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:10.178445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffbc05-7173-4b5a-9306-5e2c60885da4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:10.278184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e88644-b98f-4c56-9f18-b7ff94ea7235 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:10.289029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffbc05-7173-4b5a-9306-5e2c60885da4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:10.305701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:10.315135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165918c7-e178-48d4-aa3c-e14074d5e990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:10.315716 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165918c7-e178-48d4-aa3c-e14074d5e990 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:10.316004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165918c7-e178-48d4-aa3c-e14074d5e990 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:10.316864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165918c7-e178-48d4-aa3c-e14074d5e990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:10.327442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165918c7-e178-48d4-aa3c-e14074d5e990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:10.328895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165918c7-e178-48d4-aa3c-e14074d5e990 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:10.329584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20053a4-bda4-4123-a4f8-44c25c3d9b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:10.330429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20053a4-bda4-4123-a4f8-44c25c3d9b75 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:10.330899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20053a4-bda4-4123-a4f8-44c25c3d9b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:10.331341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20053a4-bda4-4123-a4f8-44c25c3d9b75 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:10.333097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20053a4-bda4-4123-a4f8-44c25c3d9b75 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:10.334609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20053a4-bda4-4123-a4f8-44c25c3d9b75 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-165918c7-e178-48d4-aa3c-e14074d5e990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:10.345239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:10.346269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:10.348068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:10.348068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:10.348068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:10.348068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:10.350203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 379/744] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:10.351101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165918c7-e178-48d4-aa3c-e14074d5e990 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:10.362844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 366/745] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:47:10 2022] POST /identity/v3/services => generated 240 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:10.362844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20053a4-bda4-4123-a4f8-44c25c3d9b75 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:10.420687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:10.484240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165918c7-e178-48d4-aa3c-e14074d5e990 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:10.525203 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20053a4-bda4-4123-a4f8-44c25c3d9b75 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:10.526503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5be18-652d-4ec9-86ab-132041ba9e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:10.527168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5be18-652d-4ec9-86ab-132041ba9e33 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:10.527448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5be18-652d-4ec9-86ab-132041ba9e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:10.527725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5be18-652d-4ec9-86ab-132041ba9e33 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:10.528430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5be18-652d-4ec9-86ab-132041ba9e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:10.528731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5be18-652d-4ec9-86ab-132041ba9e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:10.530111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5be18-652d-4ec9-86ab-132041ba9e33 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:10.548516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 380/746] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:10.549565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5be18-652d-4ec9-86ab-132041ba9e33 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:10.576307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dd912-dd8c-457b-85f8-dc02c0d29aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:10.576945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dd912-dd8c-457b-85f8-dc02c0d29aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:10.577232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dd912-dd8c-457b-85f8-dc02c0d29aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:10.577506 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dd912-dd8c-457b-85f8-dc02c0d29aa2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:10.589987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dd912-dd8c-457b-85f8-dc02c0d29aa2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:10.592221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dd912-dd8c-457b-85f8-dc02c0d29aa2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:10.600728 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-997dd912-dd8c-457b-85f8-dc02c0d29aa2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:10.605252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:10.606229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 367/747] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jul 28 03:47:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:10.608127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dd912-dd8c-457b-85f8-dc02c0d29aa2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:10.673446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5be18-652d-4ec9-86ab-132041ba9e33 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:10.709342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42b7a7-404b-4532-8263-fdb56e8e4c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:10.709967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42b7a7-404b-4532-8263-fdb56e8e4c1c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:10.710250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42b7a7-404b-4532-8263-fdb56e8e4c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:10.710529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42b7a7-404b-4532-8263-fdb56e8e4c1c None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:10.711374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42b7a7-404b-4532-8263-fdb56e8e4c1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:10.711712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42b7a7-404b-4532-8263-fdb56e8e4c1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:10.713474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42b7a7-404b-4532-8263-fdb56e8e4c1c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:10.727268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 381/748] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:10.728044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42b7a7-404b-4532-8263-fdb56e8e4c1c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:10.735671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:10.749934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dd912-dd8c-457b-85f8-dc02c0d29aa2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:10.794251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977bc90-0189-413f-a934-31cf7a9ef6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:10.795027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bc90-0189-413f-a934-31cf7a9ef6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:10.795214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bc90-0189-413f-a934-31cf7a9ef6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:10.795799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bc90-0189-413f-a934-31cf7a9ef6b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:10.805828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977bc90-0189-413f-a934-31cf7a9ef6b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:10.808406 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977bc90-0189-413f-a934-31cf7a9ef6b3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-9977bc90-0189-413f-a934-31cf7a9ef6b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:10.824623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:10.825526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:10.826462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:10.826462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:10.826462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:10.826462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:10.826462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:10.826462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:10.826462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:10.826462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:10.833729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 368/749] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:10.833729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977bc90-0189-413f-a934-31cf7a9ef6b3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:10.855060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:10.861840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42b7a7-404b-4532-8263-fdb56e8e4c1c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:10.912855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c0f0b-2335-46a5-b413-3e7d4e4563d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:10.914687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c0f0b-2335-46a5-b413-3e7d4e4563d1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:10.915146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c0f0b-2335-46a5-b413-3e7d4e4563d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:10.915570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c0f0b-2335-46a5-b413-3e7d4e4563d1 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:10.916951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c0f0b-2335-46a5-b413-3e7d4e4563d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:10.917824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c0f0b-2335-46a5-b413-3e7d4e4563d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:10.919750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c0f0b-2335-46a5-b413-3e7d4e4563d1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:10.940910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977bc90-0189-413f-a934-31cf7a9ef6b3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:10.946411 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 382/750] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jul 28 03:47:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:10.947633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c0f0b-2335-46a5-b413-3e7d4e4563d1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:10.979629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516ff75-32a0-4930-869d-ef88da1cae02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:10.980266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516ff75-32a0-4930-869d-ef88da1cae02 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:10.980616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516ff75-32a0-4930-869d-ef88da1cae02 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:10.980980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516ff75-32a0-4930-869d-ef88da1cae02 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:10.989406 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516ff75-32a0-4930-869d-ef88da1cae02 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:10.991026 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516ff75-32a0-4930-869d-ef88da1cae02 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c516ff75-32a0-4930-869d-ef88da1cae02 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:11.003326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:11.004416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:11.005368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:11.005368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:11.005368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:11.005368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:11.005368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:11.005368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:11.005368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:11.005368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:11.005368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:47:11.005368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:11.009863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 369/751] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:11.010663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516ff75-32a0-4930-869d-ef88da1cae02 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:11.027450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:11.092013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c0f0b-2335-46a5-b413-3e7d4e4563d1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:11.120014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516ff75-32a0-4930-869d-ef88da1cae02 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:11.131987 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f243d66-b875-459c-a583-fb7db31a7a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:11.132941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f243d66-b875-459c-a583-fb7db31a7a11 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:11.133443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f243d66-b875-459c-a583-fb7db31a7a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:11.133974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f243d66-b875-459c-a583-fb7db31a7a11 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:11.135082 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f243d66-b875-459c-a583-fb7db31a7a11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:11.135622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f243d66-b875-459c-a583-fb7db31a7a11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:11.137444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f243d66-b875-459c-a583-fb7db31a7a11 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:11.157091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 383/752] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:11.157091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f243d66-b875-459c-a583-fb7db31a7a11 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:11.161141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04687fae-80a9-4758-b2f4-cddb9670d114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:11.162031 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687fae-80a9-4758-b2f4-cddb9670d114 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:11.162307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687fae-80a9-4758-b2f4-cddb9670d114 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:11.162637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687fae-80a9-4758-b2f4-cddb9670d114 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:11.180255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:11.236646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04687fae-80a9-4758-b2f4-cddb9670d114 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:11.238347 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04687fae-80a9-4758-b2f4-cddb9670d114 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:11.253225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:11.274241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f243d66-b875-459c-a583-fb7db31a7a11 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:11.313582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb84ab-3853-4c4e-999f-df8e3aec74c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:11.315378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb84ab-3853-4c4e-999f-df8e3aec74c3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:11.315897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb84ab-3853-4c4e-999f-df8e3aec74c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:11.317722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb84ab-3853-4c4e-999f-df8e3aec74c3 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:11.318885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb84ab-3853-4c4e-999f-df8e3aec74c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:11.319408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb84ab-3853-4c4e-999f-df8e3aec74c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:11.321137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb84ab-3853-4c4e-999f-df8e3aec74c3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:11.340797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 384/753] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jul 28 03:47:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:11.342095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb84ab-3853-4c4e-999f-df8e3aec74c3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:11.350263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 370/754] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:47:11.351246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04687fae-80a9-4758-b2f4-cddb9670d114 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:11.424433 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:11.454307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb84ab-3853-4c4e-999f-df8e3aec74c3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:11.461672 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04687fae-80a9-4758-b2f4-cddb9670d114 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:11.480515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:11.490554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa39ae-8d1f-4811-b5cc-dc8405a81bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:11.491255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa39ae-8d1f-4811-b5cc-dc8405a81bd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:11.491533 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa39ae-8d1f-4811-b5cc-dc8405a81bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:11.491805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa39ae-8d1f-4811-b5cc-dc8405a81bd2 None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/162ba16d643747c287bfe8a8e59df235/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:11.493726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa39ae-8d1f-4811-b5cc-dc8405a81bd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=162ba16d643747c287bfe8a8e59df235, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:11.495177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa39ae-8d1f-4811-b5cc-dc8405a81bd2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:11.510050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177ecb9-959c-4cff-8971-169a10149794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:11.511046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177ecb9-959c-4cff-8971-169a10149794 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:11.511556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177ecb9-959c-4cff-8971-169a10149794 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:11.518812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 385/755] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:47:11 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/162ba16d643747c287bfe8a8e59df235/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:47:11.518812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa39ae-8d1f-4811-b5cc-dc8405a81bd2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:11.519115 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177ecb9-959c-4cff-8971-169a10149794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:11.663352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177ecb9-959c-4cff-8971-169a10149794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:11.664765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177ecb9-959c-4cff-8971-169a10149794 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:11.720602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa39ae-8d1f-4811-b5cc-dc8405a81bd2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:11.763350 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cb28f-178c-4b15-8276-c38ddf8990a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:11.764003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cb28f-178c-4b15-8276-c38ddf8990a7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:11.764288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cb28f-178c-4b15-8276-c38ddf8990a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:11.764560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cb28f-178c-4b15-8276-c38ddf8990a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:11.766942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:11.777477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1cb28f-178c-4b15-8276-c38ddf8990a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:11.780208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1cb28f-178c-4b15-8276-c38ddf8990a7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:11.794244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 371/756] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-6f1cb28f-178c-4b15-8276-c38ddf8990a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:11.797579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:11.799109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:11.800018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:11.800018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:11.800018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:11.800018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:11.800018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 386/757] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:11.800301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177ecb9-959c-4cff-8971-169a10149794 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:11.808378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cb28f-178c-4b15-8276-c38ddf8990a7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:11.813736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:11.919580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cb28f-178c-4b15-8276-c38ddf8990a7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:11.956362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023316c-caa1-4095-99af-970a2811aa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:11.957644 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023316c-caa1-4095-99af-970a2811aa0e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:11.957896 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023316c-caa1-4095-99af-970a2811aa0e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:11.958261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023316c-caa1-4095-99af-970a2811aa0e None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:11.959104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023316c-caa1-4095-99af-970a2811aa0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:11.959417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023316c-caa1-4095-99af-970a2811aa0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:11.960788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023316c-caa1-4095-99af-970a2811aa0e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:11.974941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 387/758] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:11.979765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023316c-caa1-4095-99af-970a2811aa0e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:12.035648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:12.065110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177ecb9-959c-4cff-8971-169a10149794 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:12.100582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023316c-caa1-4095-99af-970a2811aa0e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:12.143916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa330d00-3194-4c0a-93c4-2baf8966c7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:12.146339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa330d00-3194-4c0a-93c4-2baf8966c7fc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.147961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa330d00-3194-4c0a-93c4-2baf8966c7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.149216 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa330d00-3194-4c0a-93c4-2baf8966c7fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.155629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae6c9d-3c1c-4373-9164-14ea88a12006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:12.160317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae6c9d-3c1c-4373-9164-14ea88a12006 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.160970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae6c9d-3c1c-4373-9164-14ea88a12006 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.161742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae6c9d-3c1c-4373-9164-14ea88a12006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.164984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa330d00-3194-4c0a-93c4-2baf8966c7fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:12.167246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa330d00-3194-4c0a-93c4-2baf8966c7fc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:12.183378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae6c9d-3c1c-4373-9164-14ea88a12006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:12.185772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae6c9d-3c1c-4373-9164-14ea88a12006 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-fa330d00-3194-4c0a-93c4-2baf8966c7fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:12.191981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:12.192953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:12.193856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:12.193856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:12.193856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:12.193856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:12.193856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:12.193856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:12.193856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:12.193856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:12.200431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 388/759] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:12.205871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa330d00-3194-4c0a-93c4-2baf8966c7fc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6bae6c9d-3c1c-4373-9164-14ea88a12006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:12.212862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:12.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:12.214678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:12.214678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:12.214678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:12.214678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:12.216912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 372/760] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:12.218173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a365e3-8f01-42f7-be65-aa6127741e24 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.218460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a365e3-8f01-42f7-be65-aa6127741e24 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.218738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a365e3-8f01-42f7-be65-aa6127741e24 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.220539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 373/761] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:12.224120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a365e3-8f01-42f7-be65-aa6127741e24 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:12.309303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:12.330962 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa330d00-3194-4c0a-93c4-2baf8966c7fc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:12.339870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:12.368909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a365e3-8f01-42f7-be65-aa6127741e24 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:12.373920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae59cb0-0bf7-4f23-b6d5-41402f0f1042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:12.374621 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae59cb0-0bf7-4f23-b6d5-41402f0f1042 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.375022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae59cb0-0bf7-4f23-b6d5-41402f0f1042 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.375370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae59cb0-0bf7-4f23-b6d5-41402f0f1042 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.376197 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae59cb0-0bf7-4f23-b6d5-41402f0f1042 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:12.376572 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae59cb0-0bf7-4f23-b6d5-41402f0f1042 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:12.378073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae59cb0-0bf7-4f23-b6d5-41402f0f1042 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:12.391379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 389/762] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:12.392818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855617a-361a-45af-bab1-94728e074240 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.393176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855617a-361a-45af-bab1-94728e074240 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.393522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855617a-361a-45af-bab1-94728e074240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.413893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0855617a-361a-45af-bab1-94728e074240 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:12.414115 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e9f1a-bdae-4c32-8b77-bcdbf090860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:12.414978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e9f1a-bdae-4c32-8b77-bcdbf090860e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.415304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0855617a-361a-45af-bab1-94728e074240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:12.415656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e9f1a-bdae-4c32-8b77-bcdbf090860e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.416120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e9f1a-bdae-4c32-8b77-bcdbf090860e None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.416994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e9f1a-bdae-4c32-8b77-bcdbf090860e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:12.417785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e9f1a-bdae-4c32-8b77-bcdbf090860e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:12.420325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e9f1a-bdae-4c32-8b77-bcdbf090860e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:12.438231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 374/763] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:12.440009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e9f1a-bdae-4c32-8b77-bcdbf090860e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:12.499029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 390/764] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:12.500218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855617a-361a-45af-bab1-94728e074240 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:12.584803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e9f1a-bdae-4c32-8b77-bcdbf090860e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:12.610225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:12.612942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:12.619482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855617a-361a-45af-bab1-94728e074240 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:12.621520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24345070-4afe-4322-adb0-386fac026713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:12.622130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24345070-4afe-4322-adb0-386fac026713 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.622418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24345070-4afe-4322-adb0-386fac026713 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.622682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24345070-4afe-4322-adb0-386fac026713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.623498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24345070-4afe-4322-adb0-386fac026713 None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=3b23317779d94f99bf804a172bb9c4ed) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:12.623804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24345070-4afe-4322-adb0-386fac026713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:12.625215 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24345070-4afe-4322-adb0-386fac026713 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:12.638817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 375/765] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:12 2022] GET /identity/v3/role_assignments?user.id=3b23317779d94f99bf804a172bb9c4ed&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:12.638817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24345070-4afe-4322-adb0-386fac026713 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:12.672947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc3ed-e7c4-4dbd-923f-163a15ff7113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:12.673967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc3ed-e7c4-4dbd-923f-163a15ff7113 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.674322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc3ed-e7c4-4dbd-923f-163a15ff7113 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.675125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc3ed-e7c4-4dbd-923f-163a15ff7113 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.684801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cc3ed-e7c4-4dbd-923f-163a15ff7113 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:12.687217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cc3ed-e7c4-4dbd-923f-163a15ff7113 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-585cc3ed-e7c4-4dbd-923f-163a15ff7113 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:12.705561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:12.706491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:12.708086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:12.708086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:12.708086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:12.708086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:12.708086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:12.708086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:12.708086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:12.708086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:12.711453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 391/766] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:12.713060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482a841-851d-4a21-9b7d-d8c5ae2d662d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.713508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482a841-851d-4a21-9b7d-d8c5ae2d662d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.714008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482a841-851d-4a21-9b7d-d8c5ae2d662d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.715806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 392/767] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:12.721879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482a841-851d-4a21-9b7d-d8c5ae2d662d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:12.772588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24345070-4afe-4322-adb0-386fac026713 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:12.812062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d4750-4ba3-4031-8789-8b38191b2139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:12.812729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d4750-4ba3-4031-8789-8b38191b2139 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.812963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d4750-4ba3-4031-8789-8b38191b2139 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.813233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d4750-4ba3-4031-8789-8b38191b2139 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.813862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d4750-4ba3-4031-8789-8b38191b2139 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:12.815250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d4750-4ba3-4031-8789-8b38191b2139 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a21d4750-4ba3-4031-8789-8b38191b2139 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:12.825036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:12.826005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:12.827502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 376/768] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Jul 28 03:47:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:12.828370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c505c3-fb19-4373-8ba3-ab2c1200dc0c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.828370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c505c3-fb19-4373-8ba3-ab2c1200dc0c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.828370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c505c3-fb19-4373-8ba3-ab2c1200dc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.846329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f3c505c3-fb19-4373-8ba3-ab2c1200dc0c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:12.848912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f3c505c3-fb19-4373-8ba3-ab2c1200dc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:12.857266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482a841-851d-4a21-9b7d-d8c5ae2d662d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:12.900420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:12.906378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b875c2-10cc-4c0f-8abf-99097107d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:12.908599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b875c2-10cc-4c0f-8abf-99097107d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:12.909092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b875c2-10cc-4c0f-8abf-99097107d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:12.910344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b875c2-10cc-4c0f-8abf-99097107d6c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:12.912166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b875c2-10cc-4c0f-8abf-99097107d6c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:12.913027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b875c2-10cc-4c0f-8abf-99097107d6c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:12.915209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b875c2-10cc-4c0f-8abf-99097107d6c0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:12.929132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 377/769] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:12.930297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c505c3-fb19-4373-8ba3-ab2c1200dc0c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:12.938918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 393/770] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:12.938918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b875c2-10cc-4c0f-8abf-99097107d6c0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:12.967272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:13.066983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c505c3-fb19-4373-8ba3-ab2c1200dc0c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:13.069464 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b875c2-10cc-4c0f-8abf-99097107d6c0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:13.107501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136868a9-7054-474a-8a56-69e30231dc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:13.108211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136868a9-7054-474a-8a56-69e30231dc95 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:13.108589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136868a9-7054-474a-8a56-69e30231dc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:13.108895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136868a9-7054-474a-8a56-69e30231dc95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:13.109855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136868a9-7054-474a-8a56-69e30231dc95 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=19779569e08a42148d2933f644e3b9fa) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:13.110257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136868a9-7054-474a-8a56-69e30231dc95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:13.112041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136868a9-7054-474a-8a56-69e30231dc95 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:13.113868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3c128-a5eb-4c99-808e-348c8ef87631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:13.114983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3c128-a5eb-4c99-808e-348c8ef87631 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:13.115439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3c128-a5eb-4c99-808e-348c8ef87631 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:13.116436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3c128-a5eb-4c99-808e-348c8ef87631 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:13.117581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3c128-a5eb-4c99-808e-348c8ef87631 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:13.118077 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3c128-a5eb-4c99-808e-348c8ef87631 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:13.124320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 394/771] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:12 2022] GET /identity/v3/role_assignments?user.id=19779569e08a42148d2933f644e3b9fa&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=a37ae24c516b48c69e13721c013134a2 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:13.124320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136868a9-7054-474a-8a56-69e30231dc95 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:13.124588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3c128-a5eb-4c99-808e-348c8ef87631 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:13.136049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 378/772] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jul 28 03:47:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:13.143707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3c128-a5eb-4c99-808e-348c8ef87631 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:13.242171 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:13.305047 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:13.333341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136868a9-7054-474a-8a56-69e30231dc95 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:13.357353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3c128-a5eb-4c99-808e-348c8ef87631 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:13.374056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fc01e-95d9-4343-a68f-fbd6111a7c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:13.374707 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fc01e-95d9-4343-a68f-fbd6111a7c94 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:13.375031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fc01e-95d9-4343-a68f-fbd6111a7c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:13.375291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fc01e-95d9-4343-a68f-fbd6111a7c94 None admin] PATH_INFO: `/v3/services/network` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:13.375943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16fc01e-95d9-4343-a68f-fbd6111a7c94 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:13.377418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16fc01e-95d9-4343-a68f-fbd6111a7c94 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e16fc01e-95d9-4343-a68f-fbd6111a7c94 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:13.388547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:13.389578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 03:47:13.390388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:13.395093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 395/773] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:13 2022] GET /identity/v3/services/network => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:13.399577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fc01e-95d9-4343-a68f-fbd6111a7c94 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:13.408589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0951ef-bd44-4e07-805a-3f04fdadc2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:13.409220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0951ef-bd44-4e07-805a-3f04fdadc2bb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:13.409500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0951ef-bd44-4e07-805a-3f04fdadc2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:13.414488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0951ef-bd44-4e07-805a-3f04fdadc2bb None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:13.415230 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0951ef-bd44-4e07-805a-3f04fdadc2bb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:13.415542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0951ef-bd44-4e07-805a-3f04fdadc2bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:13.419389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0951ef-bd44-4e07-805a-3f04fdadc2bb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:13.428677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 379/774] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jul 28 03:47:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:13.440573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0951ef-bd44-4e07-805a-3f04fdadc2bb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:13.546633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fc01e-95d9-4343-a68f-fbd6111a7c94 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:13.585736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895949f-0dfe-4161-a70b-de431f58836f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:13.586406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895949f-0dfe-4161-a70b-de431f58836f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:13.586694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895949f-0dfe-4161-a70b-de431f58836f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:13.586999 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895949f-0dfe-4161-a70b-de431f58836f None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:13.587688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895949f-0dfe-4161-a70b-de431f58836f None admin] RBAC: Adding query filter params (name=network) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:13.587995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895949f-0dfe-4161-a70b-de431f58836f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:13.589371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895949f-0dfe-4161-a70b-de431f58836f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:13.594509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 396/775] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:13.601800 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895949f-0dfe-4161-a70b-de431f58836f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:13.613568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:13.638615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0951ef-bd44-4e07-805a-3f04fdadc2bb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:13.649556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:13.688414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4f77e-d01f-4f72-9622-eb9daa7e4b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:13.689429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4f77e-d01f-4f72-9622-eb9daa7e4b19 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:13.689909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4f77e-d01f-4f72-9622-eb9daa7e4b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:13.690370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4f77e-d01f-4f72-9622-eb9daa7e4b19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:13.691917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4f77e-d01f-4f72-9622-eb9daa7e4b19 None admin] RBAC: Adding query filter params (interface=public, service_id=43361e19540e45eeac5b470114d7b077, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:13.692448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4f77e-d01f-4f72-9622-eb9daa7e4b19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:13.694017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4f77e-d01f-4f72-9622-eb9daa7e4b19 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:13.702495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 380/776] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jul 28 03:47:13 2022] GET /identity/v3/endpoints?service_id=43361e19540e45eeac5b470114d7b077&interface=public®ion_id=RegionOne => generated 194 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:13.708289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4f77e-d01f-4f72-9622-eb9daa7e4b19 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:13.718368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895949f-0dfe-4161-a70b-de431f58836f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:13.755255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eff9a-2257-48fa-89d4-66e715ab4d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:13.755891 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eff9a-2257-48fa-89d4-66e715ab4d79 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:13.756180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eff9a-2257-48fa-89d4-66e715ab4d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:13.756497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eff9a-2257-48fa-89d4-66e715ab4d79 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:13.757239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406eff9a-2257-48fa-89d4-66e715ab4d79 None admin] RBAC: Adding query filter params (type=network) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:13.757549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406eff9a-2257-48fa-89d4-66e715ab4d79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:13.758931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406eff9a-2257-48fa-89d4-66e715ab4d79 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:13.765731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 397/777] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:13.771989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eff9a-2257-48fa-89d4-66e715ab4d79 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:13.817952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4f77e-d01f-4f72-9622-eb9daa7e4b19 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:13.851814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:13.857947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141896e1-65ee-49f6-9df4-4a263afef6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:13.858574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141896e1-65ee-49f6-9df4-4a263afef6cb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:13.858907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141896e1-65ee-49f6-9df4-4a263afef6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:13.859186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141896e1-65ee-49f6-9df4-4a263afef6cb None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:13.859846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141896e1-65ee-49f6-9df4-4a263afef6cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:13.861226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141896e1-65ee-49f6-9df4-4a263afef6cb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:13.869101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 381/778] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jul 28 03:47:13 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:13.870651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d18475-9b7f-4679-853f-c6b9498ec502 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:13.870979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d18475-9b7f-4679-853f-c6b9498ec502 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:13.871257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d18475-9b7f-4679-853f-c6b9498ec502 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:13.872795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 382/779] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:13.880641 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16637a-c436-4e81-9ac9-ea3c054ab0de None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:13.880935 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16637a-c436-4e81-9ac9-ea3c054ab0de None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:13.881218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16637a-c436-4e81-9ac9-ea3c054ab0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:13.887776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eff9a-2257-48fa-89d4-66e715ab4d79 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:13.902108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ba16637a-c436-4e81-9ac9-ea3c054ab0de None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:13.903567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ba16637a-c436-4e81-9ac9-ea3c054ab0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:13.924719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d96dcf-8c72-4738-a165-f531a797fe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:13.925886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d96dcf-8c72-4738-a165-f531a797fe71 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:13.926172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d96dcf-8c72-4738-a165-f531a797fe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:13.926450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d96dcf-8c72-4738-a165-f531a797fe71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:13.927253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d96dcf-8c72-4738-a165-f531a797fe71 None admin] RBAC: Adding query filter params (interface=public, service_id=b36ff9d0ff554efeb76b4f361a6dce15, region_id=RegionOne) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:13.927566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d96dcf-8c72-4738-a165-f531a797fe71 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:13.928978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d96dcf-8c72-4738-a165-f531a797fe71 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:13.937693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 398/780] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jul 28 03:47:13 2022] GET /identity/v3/endpoints?service_id=b36ff9d0ff554efeb76b4f361a6dce15&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:13.943844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d96dcf-8c72-4738-a165-f531a797fe71 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:13.949760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:14.029264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 383/781] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:14.063388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16637a-c436-4e81-9ac9-ea3c054ab0de None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:14.083029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d96dcf-8c72-4738-a165-f531a797fe71 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:14.119264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efded0f-64c5-4f8f-b75d-50f49db26657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:14.119896 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efded0f-64c5-4f8f-b75d-50f49db26657 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:14.120191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efded0f-64c5-4f8f-b75d-50f49db26657 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:14.120471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efded0f-64c5-4f8f-b75d-50f49db26657 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:14.121131 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efded0f-64c5-4f8f-b75d-50f49db26657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:14.122510 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efded0f-64c5-4f8f-b75d-50f49db26657 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:14.124528 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:14.134995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 399/782] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jul 28 03:47:13 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:14.202128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16637a-c436-4e81-9ac9-ea3c054ab0de None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:14.243509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27f5fd-cdb4-4353-a26a-999c2abbba61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:14.244169 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27f5fd-cdb4-4353-a26a-999c2abbba61 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:14.244506 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27f5fd-cdb4-4353-a26a-999c2abbba61 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:14.244792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27f5fd-cdb4-4353-a26a-999c2abbba61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:14.257226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27f5fd-cdb4-4353-a26a-999c2abbba61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6e27f5fd-cdb4-4353-a26a-999c2abbba61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:14.262305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:14.264515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:14.266478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 384/783] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:14.272166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efded0f-64c5-4f8f-b75d-50f49db26657 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:14.396372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efded0f-64c5-4f8f-b75d-50f49db26657 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:14.431596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e0b3f-eb26-4a37-868b-e0a4ad81c00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:14.432205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e0b3f-eb26-4a37-868b-e0a4ad81c00b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:14.432489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e0b3f-eb26-4a37-868b-e0a4ad81c00b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:14.432767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e0b3f-eb26-4a37-868b-e0a4ad81c00b None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:14.433490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e0b3f-eb26-4a37-868b-e0a4ad81c00b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:14.433797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e0b3f-eb26-4a37-868b-e0a4ad81c00b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:14.435296 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e0b3f-eb26-4a37-868b-e0a4ad81c00b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:14.445026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 400/784] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:47:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:14.450211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27f5fd-cdb4-4353-a26a-999c2abbba61 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:14.491779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:14.577461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27f5fd-cdb4-4353-a26a-999c2abbba61 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:14.615724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5002e-a6c4-4174-aee6-4178c18400aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:14.616371 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5002e-a6c4-4174-aee6-4178c18400aa None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:14.616659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5002e-a6c4-4174-aee6-4178c18400aa None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:14.616934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5002e-a6c4-4174-aee6-4178c18400aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:14.634743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5002e-a6c4-4174-aee6-4178c18400aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:14.636332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5002e-a6c4-4174-aee6-4178c18400aa None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b4c5002e-a6c4-4174-aee6-4178c18400aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:14.653532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:14.655618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:14.657681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:14.657681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:14.657681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:14.657681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:14.657681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 385/785] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:14.660788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e0b3f-eb26-4a37-868b-e0a4ad81c00b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:14.752088 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:14.797484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e0b3f-eb26-4a37-868b-e0a4ad81c00b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:14.827370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db648e-2761-4e00-af59-624f8587f2b2 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:14.827878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db648e-2761-4e00-af59-624f8587f2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:14.827878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db648e-2761-4e00-af59-624f8587f2b2 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:14.829691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 386/786] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:47:14.835402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dade17-7dc0-452a-98bc-41bee68a278a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:14.835752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dade17-7dc0-452a-98bc-41bee68a278a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:14.836096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dade17-7dc0-452a-98bc-41bee68a278a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:14.838128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc59d74-2742-4ad8-83f3-1adfb9dfe142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:14.838757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc59d74-2742-4ad8-83f3-1adfb9dfe142 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:14.839150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc59d74-2742-4ad8-83f3-1adfb9dfe142 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:14.839432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc59d74-2742-4ad8-83f3-1adfb9dfe142 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:14.840138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc59d74-2742-4ad8-83f3-1adfb9dfe142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:14.840444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc59d74-2742-4ad8-83f3-1adfb9dfe142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:14.841807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc59d74-2742-4ad8-83f3-1adfb9dfe142 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:14.858829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 401/787] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:14.859734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc59d74-2742-4ad8-83f3-1adfb9dfe142 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:14.891810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-67dade17-7dc0-452a-98bc-41bee68a278a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:14.893004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-67dade17-7dc0-452a-98bc-41bee68a278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:15.016601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc59d74-2742-4ad8-83f3-1adfb9dfe142 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:15.028801 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 387/788] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:15.051166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:15.058194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dade17-7dc0-452a-98bc-41bee68a278a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:15.072912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:15.081076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e837215-83e1-4d28-b632-d6aa3da8906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:15.081783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e837215-83e1-4d28-b632-d6aa3da8906b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:15.082161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e837215-83e1-4d28-b632-d6aa3da8906b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:15.082444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e837215-83e1-4d28-b632-d6aa3da8906b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:15.097823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e837215-83e1-4d28-b632-d6aa3da8906b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:15.100595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e837215-83e1-4d28-b632-d6aa3da8906b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-7e837215-83e1-4d28-b632-d6aa3da8906b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:15.128230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:15.129583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:15.130525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:15.130525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:15.130525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:15.130525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:15.130525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:15.130525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:15.130525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:15.130525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:15.130525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:15.130525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:15.131903 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 402/789] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jul 28 03:47:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:15.138708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e837215-83e1-4d28-b632-d6aa3da8906b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:15.250660 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dade17-7dc0-452a-98bc-41bee68a278a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:15.321724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b149218-7a82-490f-b13a-bbaa258dd75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:15.322375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149218-7a82-490f-b13a-bbaa258dd75d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:15.322661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149218-7a82-490f-b13a-bbaa258dd75d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:15.322962 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149218-7a82-490f-b13a-bbaa258dd75d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:15.335085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e837215-83e1-4d28-b632-d6aa3da8906b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:15.351379 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b149218-7a82-490f-b13a-bbaa258dd75d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2b149218-7a82-490f-b13a-bbaa258dd75d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:15.356462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:15.357542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:47:15.358369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:15.363941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 388/790] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:47:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:15.364889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b149218-7a82-490f-b13a-bbaa258dd75d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:15.405135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9eb12c-c202-4a17-901e-0d3c5e0e3a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:15.405756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9eb12c-c202-4a17-901e-0d3c5e0e3a38 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:15.406040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9eb12c-c202-4a17-901e-0d3c5e0e3a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:15.406315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9eb12c-c202-4a17-901e-0d3c5e0e3a38 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:15.407316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9eb12c-c202-4a17-901e-0d3c5e0e3a38 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:15.408505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9eb12c-c202-4a17-901e-0d3c5e0e3a38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:15.412537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9eb12c-c202-4a17-901e-0d3c5e0e3a38 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:15.450464 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 403/791] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jul 28 03:47:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:15.456288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9eb12c-c202-4a17-901e-0d3c5e0e3a38 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:15.474913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:15.533236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b149218-7a82-490f-b13a-bbaa258dd75d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:15.571364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:15.615306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9eb12c-c202-4a17-901e-0d3c5e0e3a38 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:15.652409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac756d-c71c-4add-9a38-5d1d998d3107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:15.653041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac756d-c71c-4add-9a38-5d1d998d3107 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:15.653331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac756d-c71c-4add-9a38-5d1d998d3107 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:15.653621 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac756d-c71c-4add-9a38-5d1d998d3107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:15.683570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd58fc-92b8-4233-9ded-4a6cd730cc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:15.684218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd58fc-92b8-4233-9ded-4a6cd730cc2c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:15.684503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd58fc-92b8-4233-9ded-4a6cd730cc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:15.684782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd58fc-92b8-4233-9ded-4a6cd730cc2c None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:15.685602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd58fc-92b8-4233-9ded-4a6cd730cc2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:15.685917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd58fc-92b8-4233-9ded-4a6cd730cc2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:15.687385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd58fc-92b8-4233-9ded-4a6cd730cc2c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:15.717112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 389/792] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:47:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:15.727838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd58fc-92b8-4233-9ded-4a6cd730cc2c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:15.729549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac756d-c71c-4add-9a38-5d1d998d3107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:15.731103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac756d-c71c-4add-9a38-5d1d998d3107 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:15.791138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:15.856374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 404/793] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 03:47:15.859996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54a0ba-99d5-480f-8132-cd40110dc008 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:15.860290 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54a0ba-99d5-480f-8132-cd40110dc008 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:15.860571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54a0ba-99d5-480f-8132-cd40110dc008 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:15.866918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 405/794] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:15.867812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54a0ba-99d5-480f-8132-cd40110dc008 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:15.887220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd58fc-92b8-4233-9ded-4a6cd730cc2c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:15.918618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:15.924005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0b914-acad-430b-b9b6-3652594a3c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:15.924776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0b914-acad-430b-b9b6-3652594a3c0e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:15.925132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0b914-acad-430b-b9b6-3652594a3c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:15.925829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0b914-acad-430b-b9b6-3652594a3c0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:15.941753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0b914-acad-430b-b9b6-3652594a3c0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:15.944675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0b914-acad-430b-b9b6-3652594a3c0e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f9d0b914-acad-430b-b9b6-3652594a3c0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:15.963395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:15.964495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:15.966107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:15.966107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:15.966107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:15.966107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:15.971039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 390/795] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:15.971039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dab8d-8a03-468f-ba1b-3b2a75a606de None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:15.971039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dab8d-8a03-468f-ba1b-3b2a75a606de None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:15.971039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dab8d-8a03-468f-ba1b-3b2a75a606de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:15.987550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-be4dab8d-8a03-468f-ba1b-3b2a75a606de None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:15.989137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-be4dab8d-8a03-468f-ba1b-3b2a75a606de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:16.007133 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54a0ba-99d5-480f-8132-cd40110dc008 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:16.054364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8b686-5b12-4c12-8cc6-0579abfdb5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:16.055336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8b686-5b12-4c12-8cc6-0579abfdb5fd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.055619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8b686-5b12-4c12-8cc6-0579abfdb5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.055900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8b686-5b12-4c12-8cc6-0579abfdb5fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.067588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:16.070083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8b686-5b12-4c12-8cc6-0579abfdb5fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:16.071610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8b686-5b12-4c12-8cc6-0579abfdb5fd None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:16.074894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 391/796] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:16.075794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dab8d-8a03-468f-ba1b-3b2a75a606de None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e9b8b686-5b12-4c12-8cc6-0579abfdb5fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:16.101937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:16.104023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:16.106073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:16.106073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:16.106073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:16.106073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:16.106073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 406/797] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:16.106073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8b686-5b12-4c12-8cc6-0579abfdb5fd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:16.201780 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dab8d-8a03-468f-ba1b-3b2a75a606de None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:16.215263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8b686-5b12-4c12-8cc6-0579abfdb5fd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:16.218093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:16.241798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e78564-5856-48b6-b797-d9043c00c9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:16.242449 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e78564-5856-48b6-b797-d9043c00c9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.242730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e78564-5856-48b6-b797-d9043c00c9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.243054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e78564-5856-48b6-b797-d9043c00c9f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.243789 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e78564-5856-48b6-b797-d9043c00c9f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:16.244098 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e78564-5856-48b6-b797-d9043c00c9f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:16.245463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e78564-5856-48b6-b797-d9043c00c9f1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:16.253188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93effdfd-ee5c-4642-937a-b5051578750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:16.253856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93effdfd-ee5c-4642-937a-b5051578750d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.254146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93effdfd-ee5c-4642-937a-b5051578750d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.254426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93effdfd-ee5c-4642-937a-b5051578750d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.257781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 392/798] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:16.258636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e78564-5856-48b6-b797-d9043c00c9f1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:16.265893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93effdfd-ee5c-4642-937a-b5051578750d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-93effdfd-ee5c-4642-937a-b5051578750d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:16.271991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:16.274064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:16.276014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:16.286808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 407/799] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:16.286808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93effdfd-ee5c-4642-937a-b5051578750d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:16.336222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:16.366380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e78564-5856-48b6-b797-d9043c00c9f1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:16.390654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93effdfd-ee5c-4642-937a-b5051578750d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:16.399648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83a66f-76cb-43d0-94e9-e3f4257a2e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:16.400267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83a66f-76cb-43d0-94e9-e3f4257a2e9b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.400554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83a66f-76cb-43d0-94e9-e3f4257a2e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.400829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83a66f-76cb-43d0-94e9-e3f4257a2e9b None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.401567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83a66f-76cb-43d0-94e9-e3f4257a2e9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:16.401870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83a66f-76cb-43d0-94e9-e3f4257a2e9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:16.403287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83a66f-76cb-43d0-94e9-e3f4257a2e9b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:16.415729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 393/800] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:16.416526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83a66f-76cb-43d0-94e9-e3f4257a2e9b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:16.433722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d83ad-532a-451b-999f-5e176f8dd440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:16.434768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d83ad-532a-451b-999f-5e176f8dd440 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.435329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d83ad-532a-451b-999f-5e176f8dd440 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.435870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d83ad-532a-451b-999f-5e176f8dd440 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.445167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d83ad-532a-451b-999f-5e176f8dd440 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:16.447003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d83ad-532a-451b-999f-5e176f8dd440 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:16.450491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-b87d83ad-532a-451b-999f-5e176f8dd440 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:16.460068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:16.461232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:16.462416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:16.462416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:16.462416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:16.462416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:16.462416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:16.462416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:16.462416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:16.462416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:16.462416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:16.462416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:16.464062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 408/801] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jul 28 03:47:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:16.465257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3de1-1de1-43ff-9cd2-03528e4e2495 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.465543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3de1-1de1-43ff-9cd2-03528e4e2495 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.465814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3de1-1de1-43ff-9cd2-03528e4e2495 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.467351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 409/802] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:16.468058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a3de1-1de1-43ff-9cd2-03528e4e2495 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:16.529731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83a66f-76cb-43d0-94e9-e3f4257a2e9b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:16.569756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ed524-e495-4036-a66c-690e125d46ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:16.570372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ed524-e495-4036-a66c-690e125d46ef None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.570651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ed524-e495-4036-a66c-690e125d46ef None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.570951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ed524-e495-4036-a66c-690e125d46ef None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.571687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ed524-e495-4036-a66c-690e125d46ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:16.572071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ed524-e495-4036-a66c-690e125d46ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:16.573704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ed524-e495-4036-a66c-690e125d46ef None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:16.588284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 394/803] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:47:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:16.588284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ed524-e495-4036-a66c-690e125d46ef None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:16.594857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a3de1-1de1-43ff-9cd2-03528e4e2495 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:16.608775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:16.632098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2b76c-c153-4667-a0ad-d1aabb6af6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:16.632742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2b76c-c153-4667-a0ad-d1aabb6af6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.633027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2b76c-c153-4667-a0ad-d1aabb6af6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.633300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2b76c-c153-4667-a0ad-d1aabb6af6b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.639880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2b76c-c153-4667-a0ad-d1aabb6af6b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:16.641327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2b76c-c153-4667-a0ad-d1aabb6af6b3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-eff2b76c-c153-4667-a0ad-d1aabb6af6b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:16.653248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:16.654247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:16.655178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:16.655178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:16.655178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:16.655178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:16.655178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:16.655178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:16.655178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:16.655178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:16.655178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 410/804] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:16.656053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68a917-b7a1-467a-81f4-de4b7b7b9f80 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.656158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68a917-b7a1-467a-81f4-de4b7b7b9f80 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.656383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68a917-b7a1-467a-81f4-de4b7b7b9f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.673540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-dc68a917-b7a1-467a-81f4-de4b7b7b9f80 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:16.674724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dc68a917-b7a1-467a-81f4-de4b7b7b9f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:16.684563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:16.691316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ed524-e495-4036-a66c-690e125d46ef None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:16.725396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968560a-5183-4db5-ac91-5de770bd1194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:16.725945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968560a-5183-4db5-ac91-5de770bd1194 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.726232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968560a-5183-4db5-ac91-5de770bd1194 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.726510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968560a-5183-4db5-ac91-5de770bd1194 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.727342 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968560a-5183-4db5-ac91-5de770bd1194 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:16.727653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968560a-5183-4db5-ac91-5de770bd1194 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:16.729076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968560a-5183-4db5-ac91-5de770bd1194 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:16.740633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 411/805] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:16.741627 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68a917-b7a1-467a-81f4-de4b7b7b9f80 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:16.758656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 395/806] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jul 28 03:47:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:16.760280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551163dc-8779-4ab3-8fcc-8c7a6424d8b9 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.760666 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551163dc-8779-4ab3-8fcc-8c7a6424d8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.761029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551163dc-8779-4ab3-8fcc-8c7a6424d8b9 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.762628 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 396/807] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:16.766864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551163dc-8779-4ab3-8fcc-8c7a6424d8b9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:16.842946 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68a917-b7a1-467a-81f4-de4b7b7b9f80 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:16.844476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:16.867931 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551163dc-8779-4ab3-8fcc-8c7a6424d8b9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:16.877944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29755b98-48eb-41ed-9666-36e773aa59ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:16.879374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29755b98-48eb-41ed-9666-36e773aa59ee None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.879660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29755b98-48eb-41ed-9666-36e773aa59ee None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.879931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29755b98-48eb-41ed-9666-36e773aa59ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.889660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29755b98-48eb-41ed-9666-36e773aa59ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:16.891124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29755b98-48eb-41ed-9666-36e773aa59ee None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:16.902641 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14dec5-9432-481b-9c09-8bf55639b1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:16.903297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14dec5-9432-481b-9c09-8bf55639b1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:16.903580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14dec5-9432-481b-9c09-8bf55639b1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:16.903857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14dec5-9432-481b-9c09-8bf55639b1a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:16.904605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14dec5-9432-481b-9c09-8bf55639b1a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:16.904916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14dec5-9432-481b-9c09-8bf55639b1a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-29755b98-48eb-41ed-9666-36e773aa59ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:16.905564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:16.906431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:16.907345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:16.907345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:16.907345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:16.907345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:16.907570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14dec5-9432-481b-9c09-8bf55639b1a8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:16.907639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 412/808] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:16.907639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29755b98-48eb-41ed-9666-36e773aa59ee None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:16.922010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 397/809] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:16.922839 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14dec5-9432-481b-9c09-8bf55639b1a8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:16.941478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:17.013425 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29755b98-48eb-41ed-9666-36e773aa59ee None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:17.027500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14dec5-9432-481b-9c09-8bf55639b1a8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:17.048241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0266dd-9c7b-46ef-ba74-33265f41b9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:17.048876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0266dd-9c7b-46ef-ba74-33265f41b9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.049160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0266dd-9c7b-46ef-ba74-33265f41b9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.049441 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0266dd-9c7b-46ef-ba74-33265f41b9d8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.050063 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0266dd-9c7b-46ef-ba74-33265f41b9d8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:17.051591 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0266dd-9c7b-46ef-ba74-33265f41b9d8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-fa0266dd-9c7b-46ef-ba74-33265f41b9d8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:17.060166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:17.061586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 03:47:17.062498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:17.063352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e35fdb-562d-45b8-b9e2-d7a2c1d3c896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:17.063457 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 413/810] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Jul 28 03:47:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:17.063537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e35fdb-562d-45b8-b9e2-d7a2c1d3c896 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.063537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e35fdb-562d-45b8-b9e2-d7a2c1d3c896 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.063537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e35fdb-562d-45b8-b9e2-d7a2c1d3c896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.063712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956fb3d-8995-4678-89ad-97af64368eb8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.063712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956fb3d-8995-4678-89ad-97af64368eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.063712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956fb3d-8995-4678-89ad-97af64368eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.080663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e35fdb-562d-45b8-b9e2-d7a2c1d3c896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:17.082626 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4956fb3d-8995-4678-89ad-97af64368eb8 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:17.083459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e35fdb-562d-45b8-b9e2-d7a2c1d3c896 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:17.083955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4956fb3d-8995-4678-89ad-97af64368eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:17.096914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-87e35fdb-562d-45b8-b9e2-d7a2c1d3c896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:17.098315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:17.099454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:17.100555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:17.100555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:17.100555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:17.100555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:17.100555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 398/811] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:17.100896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e35fdb-562d-45b8-b9e2-d7a2c1d3c896 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:17.157934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 414/812] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:17.158875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956fb3d-8995-4678-89ad-97af64368eb8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:17.171297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:17.207917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e35fdb-562d-45b8-b9e2-d7a2c1d3c896 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:17.241491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994eac26-b73c-4496-aa85-60c92ba612e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:17.242103 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eac26-b73c-4496-aa85-60c92ba612e0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.242383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eac26-b73c-4496-aa85-60c92ba612e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.242657 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eac26-b73c-4496-aa85-60c92ba612e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.243381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994eac26-b73c-4496-aa85-60c92ba612e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:17.243691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994eac26-b73c-4496-aa85-60c92ba612e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:17.245050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994eac26-b73c-4496-aa85-60c92ba612e0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:17.259515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 399/813] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:17.260305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994eac26-b73c-4496-aa85-60c92ba612e0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:17.263416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956fb3d-8995-4678-89ad-97af64368eb8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:17.297779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfce32-f6fa-4d1f-9f10-60b26398501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:17.298399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfce32-f6fa-4d1f-9f10-60b26398501f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.298680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfce32-f6fa-4d1f-9f10-60b26398501f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.298982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfce32-f6fa-4d1f-9f10-60b26398501f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.299767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfce32-f6fa-4d1f-9f10-60b26398501f None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=162ba16d643747c287bfe8a8e59df235) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:17.300076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfce32-f6fa-4d1f-9f10-60b26398501f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:17.302223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfce32-f6fa-4d1f-9f10-60b26398501f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:17.307854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 415/814] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:17 2022] GET /identity/v3/role_assignments?user.id=162ba16d643747c287bfe8a8e59df235&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=a37ae24c516b48c69e13721c013134a2 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:17.308627 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfce32-f6fa-4d1f-9f10-60b26398501f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:17.324329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:17.386750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994eac26-b73c-4496-aa85-60c92ba612e0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:17.417405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:17.426067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfce32-f6fa-4d1f-9f10-60b26398501f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:17.428598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85a969-ffc8-46cf-a64a-87f718526aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:17.429667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a969-ffc8-46cf-a64a-87f718526aca None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.429667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a969-ffc8-46cf-a64a-87f718526aca None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.429877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a969-ffc8-46cf-a64a-87f718526aca None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.430456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85a969-ffc8-46cf-a64a-87f718526aca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:17.430763 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85a969-ffc8-46cf-a64a-87f718526aca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:17.432193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85a969-ffc8-46cf-a64a-87f718526aca None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:17.443036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 400/815] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jul 28 03:47:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:17.443885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85a969-ffc8-46cf-a64a-87f718526aca None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:17.465639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56d046-a02f-4b31-9ecf-f0784cd64ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:17.466649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d046-a02f-4b31-9ecf-f0784cd64ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.466649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d046-a02f-4b31-9ecf-f0784cd64ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.466847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d046-a02f-4b31-9ecf-f0784cd64ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.467733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56d046-a02f-4b31-9ecf-f0784cd64ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:17.467838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56d046-a02f-4b31-9ecf-f0784cd64ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:17.469620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56d046-a02f-4b31-9ecf-f0784cd64ad8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:17.484131 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 416/816] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:17.484813 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56d046-a02f-4b31-9ecf-f0784cd64ad8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:17.563247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85a969-ffc8-46cf-a64a-87f718526aca None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:17.570716 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:17.600846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da37008-d4c9-4b8e-a512-7b961b108f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:17.601462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da37008-d4c9-4b8e-a512-7b961b108f59 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.601747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da37008-d4c9-4b8e-a512-7b961b108f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.602023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da37008-d4c9-4b8e-a512-7b961b108f59 None admin] PATH_INFO: `/v3/services/network` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.602624 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56d046-a02f-4b31-9ecf-f0784cd64ad8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:17.602726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da37008-d4c9-4b8e-a512-7b961b108f59 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:17.604068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da37008-d4c9-4b8e-a512-7b961b108f59 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2da37008-d4c9-4b8e-a512-7b961b108f59 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:17.613517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:17.614719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:17.616146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 401/817] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:17 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:17.619999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da37008-d4c9-4b8e-a512-7b961b108f59 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:17.641667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd3cb7-9c49-4c0d-8dcd-016a98900354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:17.642312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd3cb7-9c49-4c0d-8dcd-016a98900354 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.642596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd3cb7-9c49-4c0d-8dcd-016a98900354 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.642917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd3cb7-9c49-4c0d-8dcd-016a98900354 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.651871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd3cb7-9c49-4c0d-8dcd-016a98900354 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:17.653417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd3cb7-9c49-4c0d-8dcd-016a98900354 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-26bd3cb7-9c49-4c0d-8dcd-016a98900354 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:17.666389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:17.667385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 417/818] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:17.672827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd3cb7-9c49-4c0d-8dcd-016a98900354 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:17.687102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:17.737593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da37008-d4c9-4b8e-a512-7b961b108f59 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:17.775479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed239710-f7b2-4d3d-83eb-361206e579e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:17.776195 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed239710-f7b2-4d3d-83eb-361206e579e8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.776543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed239710-f7b2-4d3d-83eb-361206e579e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.776895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed239710-f7b2-4d3d-83eb-361206e579e8 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.777649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed239710-f7b2-4d3d-83eb-361206e579e8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:17.778028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed239710-f7b2-4d3d-83eb-361206e579e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:17.779491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed239710-f7b2-4d3d-83eb-361206e579e8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:17.782178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd3cb7-9c49-4c0d-8dcd-016a98900354 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:17.785824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 402/819] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jul 28 03:47:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:17.786966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed239710-f7b2-4d3d-83eb-361206e579e8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:17.819711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:17.826263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02618488-07b1-4dea-87a2-1cd779bc2582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:17.826942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02618488-07b1-4dea-87a2-1cd779bc2582 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.827225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02618488-07b1-4dea-87a2-1cd779bc2582 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.827494 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02618488-07b1-4dea-87a2-1cd779bc2582 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.835592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02618488-07b1-4dea-87a2-1cd779bc2582 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:17.837049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02618488-07b1-4dea-87a2-1cd779bc2582 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-02618488-07b1-4dea-87a2-1cd779bc2582 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:17.849221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:17.850318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:17.851830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:17.851830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:17.851830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:17.851830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:17.851830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:17.851830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:17.851830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:17.851830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:17.855261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 418/820] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:17.855927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02618488-07b1-4dea-87a2-1cd779bc2582 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:17.899347 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed239710-f7b2-4d3d-83eb-361206e579e8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:17.919391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:17.937170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b2f09-e71f-4c38-9d04-183fba17e08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:17.937776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b2f09-e71f-4c38-9d04-183fba17e08c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:17.938055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b2f09-e71f-4c38-9d04-183fba17e08c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:17.938332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b2f09-e71f-4c38-9d04-183fba17e08c None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:17.939036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b2f09-e71f-4c38-9d04-183fba17e08c None admin] RBAC: Adding query filter params (name=network) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:17.939336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b2f09-e71f-4c38-9d04-183fba17e08c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:17.940686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b2f09-e71f-4c38-9d04-183fba17e08c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:17.946121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 403/821] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:17.946924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b2f09-e71f-4c38-9d04-183fba17e08c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:17.971576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02618488-07b1-4dea-87a2-1cd779bc2582 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:18.006947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30b3ed-8d81-451f-b564-b6fc02de9373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:18.007566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30b3ed-8d81-451f-b564-b6fc02de9373 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:18.007847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30b3ed-8d81-451f-b564-b6fc02de9373 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:18.008121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30b3ed-8d81-451f-b564-b6fc02de9373 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:18.008906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30b3ed-8d81-451f-b564-b6fc02de9373 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:18.009215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30b3ed-8d81-451f-b564-b6fc02de9373 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:18.010643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30b3ed-8d81-451f-b564-b6fc02de9373 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:18.028434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 419/822] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jul 28 03:47:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:18.029243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30b3ed-8d81-451f-b564-b6fc02de9373 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:18.059956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b2f09-e71f-4c38-9d04-183fba17e08c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:18.088641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:18.098319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc041-b704-4d4d-aa93-86b51c5b366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:18.098938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc041-b704-4d4d-aa93-86b51c5b366a None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:18.099223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc041-b704-4d4d-aa93-86b51c5b366a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:18.099670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc041-b704-4d4d-aa93-86b51c5b366a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:18.100189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bc041-b704-4d4d-aa93-86b51c5b366a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:18.101549 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bc041-b704-4d4d-aa93-86b51c5b366a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:18.125887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 404/823] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:47:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:18.127058 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc041-b704-4d4d-aa93-86b51c5b366a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:18.154490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30b3ed-8d81-451f-b564-b6fc02de9373 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:18.183005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:18.206696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc659a1c-2eac-4f02-a43d-6120c5fe3281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:18.207558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc659a1c-2eac-4f02-a43d-6120c5fe3281 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:18.207936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc659a1c-2eac-4f02-a43d-6120c5fe3281 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:18.208305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc659a1c-2eac-4f02-a43d-6120c5fe3281 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:18.209160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc659a1c-2eac-4f02-a43d-6120c5fe3281 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:18.209561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc659a1c-2eac-4f02-a43d-6120c5fe3281 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:18.211146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc659a1c-2eac-4f02-a43d-6120c5fe3281 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:18.230416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 420/824] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:18.230416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc659a1c-2eac-4f02-a43d-6120c5fe3281 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:18.267959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc041-b704-4d4d-aa93-86b51c5b366a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:18.307052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32ee49-814f-426e-82fd-317c4b951309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:18.307818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32ee49-814f-426e-82fd-317c4b951309 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:18.308194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32ee49-814f-426e-82fd-317c4b951309 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:18.308569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32ee49-814f-426e-82fd-317c4b951309 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:18.309357 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32ee49-814f-426e-82fd-317c4b951309 None admin] RBAC: Adding query filter params (type=network) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:18.309736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32ee49-814f-426e-82fd-317c4b951309 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:18.311252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32ee49-814f-426e-82fd-317c4b951309 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:18.318498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 405/825] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:18.319397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32ee49-814f-426e-82fd-317c4b951309 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:18.349243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc659a1c-2eac-4f02-a43d-6120c5fe3281 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:18.357569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:18.392611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c761d-b7f4-43c8-9e5d-631409d190e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:18.393195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c761d-b7f4-43c8-9e5d-631409d190e9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:18.393473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c761d-b7f4-43c8-9e5d-631409d190e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:18.393745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c761d-b7f4-43c8-9e5d-631409d190e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:18.411695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c761d-b7f4-43c8-9e5d-631409d190e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:18.413331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c761d-b7f4-43c8-9e5d-631409d190e9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1c4c761d-b7f4-43c8-9e5d-631409d190e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:18.430035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:18.431024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:18.431932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:18.431932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:18.431932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:18.431932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:18.431932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 421/826] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:18.432607 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c761d-b7f4-43c8-9e5d-631409d190e9 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:18.438579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32ee49-814f-426e-82fd-317c4b951309 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:18.440437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:18.477123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471a6e6-88d3-448a-b91f-86236baf631f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:18.478172 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471a6e6-88d3-448a-b91f-86236baf631f None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:18.478663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471a6e6-88d3-448a-b91f-86236baf631f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:18.479156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471a6e6-88d3-448a-b91f-86236baf631f None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/3b23317779d94f99bf804a172bb9c4ed/roles/f5a9eb03e88a4e66bb9a8b6180009f72` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:18.495424 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471a6e6-88d3-448a-b91f-86236baf631f None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=3b23317779d94f99bf804a172bb9c4ed, role_id=f5a9eb03e88a4e66bb9a8b6180009f72)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:18.497186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471a6e6-88d3-448a-b91f-86236baf631f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:18.513586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 406/827] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:47:18 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/3b23317779d94f99bf804a172bb9c4ed/roles/f5a9eb03e88a4e66bb9a8b6180009f72 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:47:18.515059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471a6e6-88d3-448a-b91f-86236baf631f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:18.584288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c761d-b7f4-43c8-9e5d-631409d190e9 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:18.637948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee20b39-b0ec-4198-a830-a476188f64b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:18.638550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee20b39-b0ec-4198-a830-a476188f64b8 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:18.642829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee20b39-b0ec-4198-a830-a476188f64b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:18.642829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee20b39-b0ec-4198-a830-a476188f64b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:18.642829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee20b39-b0ec-4198-a830-a476188f64b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:18.642829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee20b39-b0ec-4198-a830-a476188f64b8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:18.662236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471a6e6-88d3-448a-b91f-86236baf631f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:18.678127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 422/828] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:47:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:18.734203 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4265c56-48ba-4e5a-b6ab-646ed99fe34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:18.734921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265c56-48ba-4e5a-b6ab-646ed99fe34a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:18.735204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265c56-48ba-4e5a-b6ab-646ed99fe34a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:18.735482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265c56-48ba-4e5a-b6ab-646ed99fe34a None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:18.736189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4265c56-48ba-4e5a-b6ab-646ed99fe34a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:18.736517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4265c56-48ba-4e5a-b6ab-646ed99fe34a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:18.737969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4265c56-48ba-4e5a-b6ab-646ed99fe34a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:18.751803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 407/829] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:18.758566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee20b39-b0ec-4198-a830-a476188f64b8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:18.781240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:18.874982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee20b39-b0ec-4198-a830-a476188f64b8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:18.912178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26935e8c-643d-427b-80ae-93aa5955f040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:18.912748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26935e8c-643d-427b-80ae-93aa5955f040 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:18.913026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26935e8c-643d-427b-80ae-93aa5955f040 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:18.913297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26935e8c-643d-427b-80ae-93aa5955f040 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:18.920539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26935e8c-643d-427b-80ae-93aa5955f040 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:18.921990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26935e8c-643d-427b-80ae-93aa5955f040 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-26935e8c-643d-427b-80ae-93aa5955f040 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:18.933748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:18.934667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:18.936083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:18.936083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:18.936083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:18.936083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:18.936083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:18.936083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:18.936083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:18.936083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:18.936083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 423/830] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:18.941759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4265c56-48ba-4e5a-b6ab-646ed99fe34a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:18.953430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:19.057331 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4265c56-48ba-4e5a-b6ab-646ed99fe34a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:19.104465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc53c8-180b-474d-81ed-1453d97367c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:19.105090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc53c8-180b-474d-81ed-1453d97367c6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:19.105370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc53c8-180b-474d-81ed-1453d97367c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:19.105642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc53c8-180b-474d-81ed-1453d97367c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:19.106464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc53c8-180b-474d-81ed-1453d97367c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:19.106776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc53c8-180b-474d-81ed-1453d97367c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:19.108293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc53c8-180b-474d-81ed-1453d97367c6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:19.126250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 408/831] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:19.133043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26935e8c-643d-427b-80ae-93aa5955f040 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:19.222067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:19.249582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26935e8c-643d-427b-80ae-93aa5955f040 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:19.285052 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70514926-219e-4a49-9e18-9655fbc1f811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:19.285727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70514926-219e-4a49-9e18-9655fbc1f811 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:19.286065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70514926-219e-4a49-9e18-9655fbc1f811 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:19.286602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70514926-219e-4a49-9e18-9655fbc1f811 None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/19779569e08a42148d2933f644e3b9fa/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:19.288604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70514926-219e-4a49-9e18-9655fbc1f811 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=19779569e08a42148d2933f644e3b9fa, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:19.290054 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70514926-219e-4a49-9e18-9655fbc1f811 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:19.304908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 424/832] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:47:19 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/19779569e08a42148d2933f644e3b9fa/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:47:19.507144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac088ee5-5d7b-412f-9736-1a0a40e7c3c4 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:19.507435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac088ee5-5d7b-412f-9736-1a0a40e7c3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:19.507631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac088ee5-5d7b-412f-9736-1a0a40e7c3c4 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:19.509340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 409/833] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:19.514587 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894d718-b4ab-4209-868e-2a3345fb2087 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:19.514901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894d718-b4ab-4209-868e-2a3345fb2087 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:19.515180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894d718-b4ab-4209-868e-2a3345fb2087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:19.541058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c894d718-b4ab-4209-868e-2a3345fb2087 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:19.541058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c894d718-b4ab-4209-868e-2a3345fb2087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:19.614948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 425/834] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:19.635376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac088ee5-5d7b-412f-9736-1a0a40e7c3c4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:19.707202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:19.742080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac088ee5-5d7b-412f-9736-1a0a40e7c3c4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:19.777520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f840a-e75c-4e47-bd17-2b1d1bbc4c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:19.778102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f840a-e75c-4e47-bd17-2b1d1bbc4c25 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:19.778387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f840a-e75c-4e47-bd17-2b1d1bbc4c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:19.778663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f840a-e75c-4e47-bd17-2b1d1bbc4c25 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:19.779322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f840a-e75c-4e47-bd17-2b1d1bbc4c25 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:19.780692 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f840a-e75c-4e47-bd17-2b1d1bbc4c25 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-614f840a-e75c-4e47-bd17-2b1d1bbc4c25 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:19.790353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:19.791445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 03:47:19.792265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:19.796486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 410/835] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Jul 28 03:47:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:19.797764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894d718-b4ab-4209-868e-2a3345fb2087 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:19.867139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:19.907787 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894d718-b4ab-4209-868e-2a3345fb2087 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:19.944851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23454016-0e02-4901-971d-430ffc101480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:19.945477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23454016-0e02-4901-971d-430ffc101480 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:19.945765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23454016-0e02-4901-971d-430ffc101480 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:19.946043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23454016-0e02-4901-971d-430ffc101480 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:19.946724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23454016-0e02-4901-971d-430ffc101480 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:19.947057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23454016-0e02-4901-971d-430ffc101480 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:19.948421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23454016-0e02-4901-971d-430ffc101480 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:19.957698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 426/836] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jul 28 03:47:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:19.959052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f840a-e75c-4e47-bd17-2b1d1bbc4c25 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:20.063583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f840a-e75c-4e47-bd17-2b1d1bbc4c25 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:20.094148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:20.099881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5ec72-081f-470c-bf38-c86eba84f09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:20.100526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ec72-081f-470c-bf38-c86eba84f09d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:20.103170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ec72-081f-470c-bf38-c86eba84f09d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:20.103453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ec72-081f-470c-bf38-c86eba84f09d None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:20.104142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5ec72-081f-470c-bf38-c86eba84f09d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:20.104522 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5ec72-081f-470c-bf38-c86eba84f09d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:20.105888 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5ec72-081f-470c-bf38-c86eba84f09d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:20.114911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 411/837] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jul 28 03:47:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:20.407246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b759c-77dd-482a-afe8-d4ed77b7d661 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:20.408230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b759c-77dd-482a-afe8-d4ed77b7d661 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:20.408710 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b759c-77dd-482a-afe8-d4ed77b7d661 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:20.411068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 427/838] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:20.416848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6edb4b-66d1-47f6-b293-cd6c2894d9d3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:20.417386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6edb4b-66d1-47f6-b293-cd6c2894d9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:20.417858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6edb4b-66d1-47f6-b293-cd6c2894d9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:20.438807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1c6edb4b-66d1-47f6-b293-cd6c2894d9d3 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:20.438807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1c6edb4b-66d1-47f6-b293-cd6c2894d9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:20.516182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 412/839] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:20.540073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b759c-77dd-482a-afe8-d4ed77b7d661 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:20.654112 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b759c-77dd-482a-afe8-d4ed77b7d661 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:20.687271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:20.689895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c88d78-cf59-4bde-9b26-c475af659773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:20.690552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c88d78-cf59-4bde-9b26-c475af659773 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:20.691148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c88d78-cf59-4bde-9b26-c475af659773 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:20.691406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c88d78-cf59-4bde-9b26-c475af659773 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:20.692328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c88d78-cf59-4bde-9b26-c475af659773 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:20.693715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c88d78-cf59-4bde-9b26-c475af659773 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-20c88d78-cf59-4bde-9b26-c475af659773 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:20.702974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:20.703927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:20.704753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 428/840] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:47:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:20.712582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6edb4b-66d1-47f6-b293-cd6c2894d9d3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:20.762263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1ea6-3578-42d3-85ab-4b3e93c1bfb8 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:20.762594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1ea6-3578-42d3-85ab-4b3e93c1bfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:20.762849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1ea6-3578-42d3-85ab-4b3e93c1bfb8 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:20.764321 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 429/841] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:20.770024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fc675-10e7-4561-9139-2703524a6137 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:20.770546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fc675-10e7-4561-9139-2703524a6137 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:20.771097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fc675-10e7-4561-9139-2703524a6137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:20.790495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a04fc675-10e7-4561-9139-2703524a6137 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:20.792108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a04fc675-10e7-4561-9139-2703524a6137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:20.827380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6edb4b-66d1-47f6-b293-cd6c2894d9d3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:20.861207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:20.874417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7a424-b5bb-4db9-8a74-9228443521e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:20.874417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7a424-b5bb-4db9-8a74-9228443521e3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:20.874417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7a424-b5bb-4db9-8a74-9228443521e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:20.874417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7a424-b5bb-4db9-8a74-9228443521e3 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:20.874847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7a424-b5bb-4db9-8a74-9228443521e3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:20.874847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7a424-b5bb-4db9-8a74-9228443521e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:20.874847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7a424-b5bb-4db9-8a74-9228443521e3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:20.875553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 430/842] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:20.880091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 413/843] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:47:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:20.885222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fc675-10e7-4561-9139-2703524a6137 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:20.892632 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7a424-b5bb-4db9-8a74-9228443521e3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:20.992730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fc675-10e7-4561-9139-2703524a6137 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:21.002484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7a424-b5bb-4db9-8a74-9228443521e3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:21.016442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:21.029043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3dd77-988f-455f-b415-4e5d78287cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:21.029762 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3dd77-988f-455f-b415-4e5d78287cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:21.030065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3dd77-988f-455f-b415-4e5d78287cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:21.030634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3dd77-988f-455f-b415-4e5d78287cd9 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:21.031428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3dd77-988f-455f-b415-4e5d78287cd9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:21.031815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3dd77-988f-455f-b415-4e5d78287cd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:21.033257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3dd77-988f-455f-b415-4e5d78287cd9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:21.039529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 431/844] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:47:20 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:21.050152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f8c9-6ae3-433a-bba7-ab923a89c1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:21.051281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f8c9-6ae3-433a-bba7-ab923a89c1ce None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:21.051566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f8c9-6ae3-433a-bba7-ab923a89c1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:21.051843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f8c9-6ae3-433a-bba7-ab923a89c1ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:21.064870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f8c9-6ae3-433a-bba7-ab923a89c1ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5627f8c9-6ae3-433a-bba7-ab923a89c1ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:21.070696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:21.071721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:21.072556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 414/845] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:47:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:21.079917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3dd77-988f-455f-b415-4e5d78287cd9 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:21.201228 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3dd77-988f-455f-b415-4e5d78287cd9 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:21.238559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa588f5c-e115-41b5-8786-ef1760f38cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:21.239209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa588f5c-e115-41b5-8786-ef1760f38cbb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:21.239496 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa588f5c-e115-41b5-8786-ef1760f38cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:21.239773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa588f5c-e115-41b5-8786-ef1760f38cbb None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:21.240501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa588f5c-e115-41b5-8786-ef1760f38cbb None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:21.240809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa588f5c-e115-41b5-8786-ef1760f38cbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:21.242213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa588f5c-e115-41b5-8786-ef1760f38cbb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:21.251252 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 432/846] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:47:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:21.257021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f8c9-6ae3-433a-bba7-ab923a89c1ce None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:21.311664 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:21.368394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f8c9-6ae3-433a-bba7-ab923a89c1ce None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:21.403894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bc020-b7a8-4a25-85ba-5b092e4c5d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:21.404807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bc020-b7a8-4a25-85ba-5b092e4c5d3d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:21.405244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bc020-b7a8-4a25-85ba-5b092e4c5d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:21.405663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bc020-b7a8-4a25-85ba-5b092e4c5d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:21.417313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bc020-b7a8-4a25-85ba-5b092e4c5d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:21.419016 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bc020-b7a8-4a25-85ba-5b092e4c5d3d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-551bc020-b7a8-4a25-85ba-5b092e4c5d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:21.434132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:21.435231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:21.436235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:21.436235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:21.436235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:21.436235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:21.437967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 415/847] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:21.443999 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa588f5c-e115-41b5-8786-ef1760f38cbb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:21.444571 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a3294-b609-4faa-98dd-448628488a9d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:21.444866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a3294-b609-4faa-98dd-448628488a9d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:21.445150 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a3294-b609-4faa-98dd-448628488a9d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:21.446711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 416/848] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:21.451520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec4f2c-809f-4163-81a4-2644d671f27e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:21.451810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec4f2c-809f-4163-81a4-2644d671f27e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:21.452085 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec4f2c-809f-4163-81a4-2644d671f27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:21.472181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6eec4f2c-809f-4163-81a4-2644d671f27e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:21.473472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6eec4f2c-809f-4163-81a4-2644d671f27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:21.513808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:21.551628 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 417/849] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:21.571025 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec4f2c-809f-4163-81a4-2644d671f27e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:21.579402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa588f5c-e115-41b5-8786-ef1760f38cbb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:21.613509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:21.621163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a7eb2-cd86-4348-bebd-75761caea29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:21.622081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a7eb2-cd86-4348-bebd-75761caea29c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:21.622575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a7eb2-cd86-4348-bebd-75761caea29c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:21.623097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a7eb2-cd86-4348-bebd-75761caea29c None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:21.623997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a7eb2-cd86-4348-bebd-75761caea29c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:21.624501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a7eb2-cd86-4348-bebd-75761caea29c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:21.626536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a7eb2-cd86-4348-bebd-75761caea29c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:21.647729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 433/850] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:21.649289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a7eb2-cd86-4348-bebd-75761caea29c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:21.708716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec4f2c-809f-4163-81a4-2644d671f27e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:21.750938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e3bfb-5e31-4a3d-92b0-e5db0d15adbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:21.751685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e3bfb-5e31-4a3d-92b0-e5db0d15adbe None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:21.751970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e3bfb-5e31-4a3d-92b0-e5db0d15adbe None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:21.752240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e3bfb-5e31-4a3d-92b0-e5db0d15adbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:21.765518 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e3bfb-5e31-4a3d-92b0-e5db0d15adbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-e84e3bfb-5e31-4a3d-92b0-e5db0d15adbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:21.771987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:21.772939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:21.773820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 418/851] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:21.779024 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e3bfb-5e31-4a3d-92b0-e5db0d15adbe None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:21.779186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a7eb2-cd86-4348-bebd-75761caea29c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:21.791577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:21.824635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da94dcd-892c-48ac-86a1-cfe4c4b6edf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:21.825488 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da94dcd-892c-48ac-86a1-cfe4c4b6edf2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:21.825863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da94dcd-892c-48ac-86a1-cfe4c4b6edf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:21.826259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da94dcd-892c-48ac-86a1-cfe4c4b6edf2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:21.834491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da94dcd-892c-48ac-86a1-cfe4c4b6edf2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:21.836183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da94dcd-892c-48ac-86a1-cfe4c4b6edf2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3da94dcd-892c-48ac-86a1-cfe4c4b6edf2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:21.849685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:21.850898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:21.851924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:21.851924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:21.851924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:21.851924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:21.851924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:21.851924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:21.851924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:21.851924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:21.851924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:21.851924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:21.856757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 434/852] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jul 28 03:47:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:21.857751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da94dcd-892c-48ac-86a1-cfe4c4b6edf2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:21.891182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:21.900836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e3bfb-5e31-4a3d-92b0-e5db0d15adbe None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:21.937223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014f12f-9d4b-4fe9-bb51-0648bc11d4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:21.938256 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014f12f-9d4b-4fe9-bb51-0648bc11d4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:21.938744 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014f12f-9d4b-4fe9-bb51-0648bc11d4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:21.939464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014f12f-9d4b-4fe9-bb51-0648bc11d4b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:21.940407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014f12f-9d4b-4fe9-bb51-0648bc11d4b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:21.940962 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014f12f-9d4b-4fe9-bb51-0648bc11d4b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:21.945325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014f12f-9d4b-4fe9-bb51-0648bc11d4b7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:21.955090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 419/853] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:47:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:21.966855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014f12f-9d4b-4fe9-bb51-0648bc11d4b7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:21.967050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da94dcd-892c-48ac-86a1-cfe4c4b6edf2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:22.008254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321391c-f323-47a8-a606-bcfaf2e3e350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:22.009128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321391c-f323-47a8-a606-bcfaf2e3e350 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:22.009618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321391c-f323-47a8-a606-bcfaf2e3e350 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:22.010040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321391c-f323-47a8-a606-bcfaf2e3e350 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:22.011109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321391c-f323-47a8-a606-bcfaf2e3e350 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:22.011572 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321391c-f323-47a8-a606-bcfaf2e3e350 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:22.013280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321391c-f323-47a8-a606-bcfaf2e3e350 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:22.033613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 435/854] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jul 28 03:47:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:22.040339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321391c-f323-47a8-a606-bcfaf2e3e350 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:22.074412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:22.100211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014f12f-9d4b-4fe9-bb51-0648bc11d4b7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:22.149683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500c792-8b79-4f3e-8f83-84b2f3806c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:22.150726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500c792-8b79-4f3e-8f83-84b2f3806c59 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:22.151174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500c792-8b79-4f3e-8f83-84b2f3806c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:22.151930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500c792-8b79-4f3e-8f83-84b2f3806c59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:22.154376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321391c-f323-47a8-a606-bcfaf2e3e350 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:22.167000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500c792-8b79-4f3e-8f83-84b2f3806c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:22.169090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500c792-8b79-4f3e-8f83-84b2f3806c59 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-9500c792-8b79-4f3e-8f83-84b2f3806c59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:22.189109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:22.190157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:22.191784 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:22.191784 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:22.191784 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:22.191784 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:22.192853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 420/855] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:22.199890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500c792-8b79-4f3e-8f83-84b2f3806c59 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:22.200900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d09e2-515d-42f6-b416-fa5a05396f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:22.201552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d09e2-515d-42f6-b416-fa5a05396f21 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:22.201831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d09e2-515d-42f6-b416-fa5a05396f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:22.202101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d09e2-515d-42f6-b416-fa5a05396f21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:22.212383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:22.282381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d09e2-515d-42f6-b416-fa5a05396f21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:22.284169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d09e2-515d-42f6-b416-fa5a05396f21 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:22.314008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:22.320104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500c792-8b79-4f3e-8f83-84b2f3806c59 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:22.356940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5653df-a7b0-4dfa-874d-c6193ca4661b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:22.357553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5653df-a7b0-4dfa-874d-c6193ca4661b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:22.357836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5653df-a7b0-4dfa-874d-c6193ca4661b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:22.358113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5653df-a7b0-4dfa-874d-c6193ca4661b None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:22.358827 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5653df-a7b0-4dfa-874d-c6193ca4661b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:22.359136 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5653df-a7b0-4dfa-874d-c6193ca4661b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:22.360502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5653df-a7b0-4dfa-874d-c6193ca4661b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:22.378193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 421/856] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:22.378193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4a02a-e70e-48a8-9b6f-9b29983a4ed8 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:22.378193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4a02a-e70e-48a8-9b6f-9b29983a4ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:22.378193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4a02a-e70e-48a8-9b6f-9b29983a4ed8 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:22.382705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 422/857] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:22.383584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4a02a-e70e-48a8-9b6f-9b29983a4ed8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:22.404619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 436/858] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:22 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:47:22.404619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d369-5860-4cd1-9b7d-50ff68539a12 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:22.404619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d369-5860-4cd1-9b7d-50ff68539a12 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:22.404619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d369-5860-4cd1-9b7d-50ff68539a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:22.420686 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2d91d369-5860-4cd1-9b7d-50ff68539a12 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:22.421863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2d91d369-5860-4cd1-9b7d-50ff68539a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:22.458484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:22.490412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 437/859] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:22.499246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4a02a-e70e-48a8-9b6f-9b29983a4ed8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:22.504245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91d369-5860-4cd1-9b7d-50ff68539a12 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:22.539425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024a571-3ebf-44b5-b0ab-8ebf012495d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:22.540179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024a571-3ebf-44b5-b0ab-8ebf012495d9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:22.540458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024a571-3ebf-44b5-b0ab-8ebf012495d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:22.540844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024a571-3ebf-44b5-b0ab-8ebf012495d9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:22.548366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024a571-3ebf-44b5-b0ab-8ebf012495d9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:22.552442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024a571-3ebf-44b5-b0ab-8ebf012495d9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7024a571-3ebf-44b5-b0ab-8ebf012495d9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:22.563563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:22.564524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:22.565454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:22.565454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:22.565454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:22.565454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:22.565454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:22.565454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:22.565454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:22.565454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:22.565454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 03:47:22.565454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:22.565454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 423/860] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:22.566587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024a571-3ebf-44b5-b0ab-8ebf012495d9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:22.578152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:22.624084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91d369-5860-4cd1-9b7d-50ff68539a12 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:22.658637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd30cb-6700-4449-b2b9-fd7e16840ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:22.659327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd30cb-6700-4449-b2b9-fd7e16840ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:22.659628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd30cb-6700-4449-b2b9-fd7e16840ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:22.659899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd30cb-6700-4449-b2b9-fd7e16840ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:22.670217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd30cb-6700-4449-b2b9-fd7e16840ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:22.671700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd30cb-6700-4449-b2b9-fd7e16840ba1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:22.676378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024a571-3ebf-44b5-b0ab-8ebf012495d9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-aedd30cb-6700-4449-b2b9-fd7e16840ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:22.686226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:22.687198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:22.688272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:22.688272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:22.688272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:22.688272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:22.692467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 438/861] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:22.693317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd30cb-6700-4449-b2b9-fd7e16840ba1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:22.705947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:22.716696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec66550-24b4-4372-bfdb-ac8e8fee33c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:22.717315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec66550-24b4-4372-bfdb-ac8e8fee33c9 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:22.717623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec66550-24b4-4372-bfdb-ac8e8fee33c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:22.717871 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec66550-24b4-4372-bfdb-ac8e8fee33c9 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:22.718560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec66550-24b4-4372-bfdb-ac8e8fee33c9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:22.720022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec66550-24b4-4372-bfdb-ac8e8fee33c9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:22.736071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 424/862] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:47:22 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:22.736791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec66550-24b4-4372-bfdb-ac8e8fee33c9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:22.833315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd30cb-6700-4449-b2b9-fd7e16840ba1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:22.844539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:22.868040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec66550-24b4-4372-bfdb-ac8e8fee33c9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:22.882916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec95f3-b8c5-461a-9434-ee1dc9f430c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:22.884221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec95f3-b8c5-461a-9434-ee1dc9f430c1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:22.884702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec95f3-b8c5-461a-9434-ee1dc9f430c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:22.885171 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec95f3-b8c5-461a-9434-ee1dc9f430c1 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:22.886284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec95f3-b8c5-461a-9434-ee1dc9f430c1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:22.887017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec95f3-b8c5-461a-9434-ee1dc9f430c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:22.888658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec95f3-b8c5-461a-9434-ee1dc9f430c1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:22.906544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17575a9-4756-4f41-876b-a9d95b66d3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:22.907200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17575a9-4756-4f41-876b-a9d95b66d3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:22.907481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17575a9-4756-4f41-876b-a9d95b66d3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:22.907753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17575a9-4756-4f41-876b-a9d95b66d3b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:22.908331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 439/863] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jul 28 03:47:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:22.908450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17575a9-4756-4f41-876b-a9d95b66d3b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:22.908735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17575a9-4756-4f41-876b-a9d95b66d3b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:22.910736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17575a9-4756-4f41-876b-a9d95b66d3b2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:22.916221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec95f3-b8c5-461a-9434-ee1dc9f430c1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:22.923933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 425/864] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:22.929773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17575a9-4756-4f41-876b-a9d95b66d3b2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:22.979129 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:23.049516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec95f3-b8c5-461a-9434-ee1dc9f430c1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:23.059046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17575a9-4756-4f41-876b-a9d95b66d3b2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:23.092444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f75575-d2b4-4dab-aae2-39b6d75b287c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:23.093581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f75575-d2b4-4dab-aae2-39b6d75b287c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:23.094013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f75575-d2b4-4dab-aae2-39b6d75b287c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:23.094677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f75575-d2b4-4dab-aae2-39b6d75b287c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:23.104249 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be29cec-8421-49d9-b1ac-03c7242020f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:23.104985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29cec-8421-49d9-b1ac-03c7242020f2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:23.105336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29cec-8421-49d9-b1ac-03c7242020f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:23.105677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29cec-8421-49d9-b1ac-03c7242020f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:23.114082 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be29cec-8421-49d9-b1ac-03c7242020f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:23.115667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be29cec-8421-49d9-b1ac-03c7242020f2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-9be29cec-8421-49d9-b1ac-03c7242020f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:23.128896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:23.129988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:23.131125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:23.131125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:23.131125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:23.131125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:23.131125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:23.131125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:23.131125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:23.131125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:23.132403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 426/865] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:23.137666 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be29cec-8421-49d9-b1ac-03c7242020f2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:23.143303 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:23.182219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f75575-d2b4-4dab-aae2-39b6d75b287c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:23.184529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f75575-d2b4-4dab-aae2-39b6d75b287c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:23.251084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be29cec-8421-49d9-b1ac-03c7242020f2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:23.252722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:23.293580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1212f-792b-4ea7-be49-e003cb4e969b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:23.294296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1212f-792b-4ea7-be49-e003cb4e969b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:23.294579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1212f-792b-4ea7-be49-e003cb4e969b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:23.294879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1212f-792b-4ea7-be49-e003cb4e969b None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:23.295627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1212f-792b-4ea7-be49-e003cb4e969b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:23.295934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1212f-792b-4ea7-be49-e003cb4e969b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:23.297481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 440/866] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:22 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:47:23.297591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1212f-792b-4ea7-be49-e003cb4e969b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:23.298750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718cf7d-e169-41c2-98f6-033eb144013a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:23.299120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718cf7d-e169-41c2-98f6-033eb144013a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:23.299401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718cf7d-e169-41c2-98f6-033eb144013a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:23.301182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 441/867] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:23.303922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718cf7d-e169-41c2-98f6-033eb144013a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:23.314622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 427/868] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:23.316712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c804-5843-4249-a89f-1346f5c9890e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:23.316712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c804-5843-4249-a89f-1346f5c9890e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:23.317144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c804-5843-4249-a89f-1346f5c9890e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:23.335130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cd28c804-5843-4249-a89f-1346f5c9890e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:23.336313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cd28c804-5843-4249-a89f-1346f5c9890e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:23.382980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:23.407588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 428/869] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:23 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:23.419661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c804-5843-4249-a89f-1346f5c9890e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:23.420438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718cf7d-e169-41c2-98f6-033eb144013a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:23.458560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade23be3-c618-41cb-b614-ed08e7df09b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:23.459236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade23be3-c618-41cb-b614-ed08e7df09b2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:23.459507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade23be3-c618-41cb-b614-ed08e7df09b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:23.459777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade23be3-c618-41cb-b614-ed08e7df09b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:23.470130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade23be3-c618-41cb-b614-ed08e7df09b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:23.472782 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade23be3-c618-41cb-b614-ed08e7df09b2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-ade23be3-c618-41cb-b614-ed08e7df09b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:23.488337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:23.489221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:23.490102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:23.490102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:23.490102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:23.490102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:23.490102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 442/870] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:23.490369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade23be3-c618-41cb-b614-ed08e7df09b2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:23.527095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:23.541353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c804-5843-4249-a89f-1346f5c9890e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:23.579992 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d0168-3e47-4b56-8624-092b339de62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:23.580702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d0168-3e47-4b56-8624-092b339de62c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:23.581059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d0168-3e47-4b56-8624-092b339de62c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:23.581965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d0168-3e47-4b56-8624-092b339de62c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:23.582873 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d0168-3e47-4b56-8624-092b339de62c None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=3b23317779d94f99bf804a172bb9c4ed) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:23.583089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d0168-3e47-4b56-8624-092b339de62c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:23.584520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d0168-3e47-4b56-8624-092b339de62c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:23.590464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 429/871] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:23 2022] GET /identity/v3/role_assignments?user.id=3b23317779d94f99bf804a172bb9c4ed&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:23.596050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d0168-3e47-4b56-8624-092b339de62c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:23.603990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade23be3-c618-41cb-b614-ed08e7df09b2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:23.624751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:23.646172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c209c-cbff-42dd-a44a-bbacae9f5951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:23.646725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c209c-cbff-42dd-a44a-bbacae9f5951 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:23.647079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c209c-cbff-42dd-a44a-bbacae9f5951 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:23.647681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c209c-cbff-42dd-a44a-bbacae9f5951 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:23.648074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c209c-cbff-42dd-a44a-bbacae9f5951 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:23.649576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c209c-cbff-42dd-a44a-bbacae9f5951 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:23.674255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 443/872] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:47:23 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:23.742603 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d0168-3e47-4b56-8624-092b339de62c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:23.790078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2f087-0066-4efb-b6d4-5e28e73053f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:23.790702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2f087-0066-4efb-b6d4-5e28e73053f7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:23.791038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2f087-0066-4efb-b6d4-5e28e73053f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:23.791307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2f087-0066-4efb-b6d4-5e28e73053f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:23.792049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2f087-0066-4efb-b6d4-5e28e73053f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:23.792332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2f087-0066-4efb-b6d4-5e28e73053f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:23.793751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2f087-0066-4efb-b6d4-5e28e73053f7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:23.809164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 430/873] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:23.814603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c209c-cbff-42dd-a44a-bbacae9f5951 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:23.924649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c209c-cbff-42dd-a44a-bbacae9f5951 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:23.967298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e1dcc-eb19-40ef-9feb-88708fceb800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:23.968225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e1dcc-eb19-40ef-9feb-88708fceb800 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:23.968662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e1dcc-eb19-40ef-9feb-88708fceb800 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:23.969087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e1dcc-eb19-40ef-9feb-88708fceb800 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:23.977365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e1dcc-eb19-40ef-9feb-88708fceb800 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:23.979044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e1dcc-eb19-40ef-9feb-88708fceb800 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-330e1dcc-eb19-40ef-9feb-88708fceb800 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:23.991426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:23.992537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:23.993670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:23.993670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:23.993670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:23.993670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:23.993670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:23.993670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:23.993670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:23.993670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:23.998878 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 444/874] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:24.004493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2f087-0066-4efb-b6d4-5e28e73053f7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:24.021228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:24.114194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2f087-0066-4efb-b6d4-5e28e73053f7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:24.150590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6970c-9a22-4879-a63b-b7e2494371d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:24.151245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6970c-9a22-4879-a63b-b7e2494371d5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:24.151529 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6970c-9a22-4879-a63b-b7e2494371d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:24.151805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6970c-9a22-4879-a63b-b7e2494371d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:24.152563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6970c-9a22-4879-a63b-b7e2494371d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:24.152872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6970c-9a22-4879-a63b-b7e2494371d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:24.154311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6970c-9a22-4879-a63b-b7e2494371d5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:24.167704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 431/875] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:24.173693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e1dcc-eb19-40ef-9feb-88708fceb800 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:24.218686 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:24.287121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e1dcc-eb19-40ef-9feb-88708fceb800 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:24.321930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2668aa-6b3a-4aa3-852f-7c5912dc7f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:24.323000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2668aa-6b3a-4aa3-852f-7c5912dc7f3e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:24.323515 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2668aa-6b3a-4aa3-852f-7c5912dc7f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:24.324039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2668aa-6b3a-4aa3-852f-7c5912dc7f3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:24.325028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2668aa-6b3a-4aa3-852f-7c5912dc7f3e None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=19779569e08a42148d2933f644e3b9fa) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:24.325542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2668aa-6b3a-4aa3-852f-7c5912dc7f3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:24.327281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2668aa-6b3a-4aa3-852f-7c5912dc7f3e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:24.338924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 445/876] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:24 2022] GET /identity/v3/role_assignments?user.id=19779569e08a42148d2933f644e3b9fa&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=a37ae24c516b48c69e13721c013134a2 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:24.848561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cb4cc-5344-48fb-a886-41fd083ef21a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:24.849266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cb4cc-5344-48fb-a886-41fd083ef21a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:24.849764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cb4cc-5344-48fb-a886-41fd083ef21a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:24.851574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 432/877] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:24.856906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188a90-e323-4545-a91e-0fe67d74074a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:24.857758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188a90-e323-4545-a91e-0fe67d74074a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:24.858305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188a90-e323-4545-a91e-0fe67d74074a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:24.877360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9f188a90-e323-4545-a91e-0fe67d74074a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:24.879202 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9f188a90-e323-4545-a91e-0fe67d74074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:24.950315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 446/878] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:24.973371 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2cb4cc-5344-48fb-a886-41fd083ef21a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:25.058833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:25.084482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2cb4cc-5344-48fb-a886-41fd083ef21a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:25.120399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd6e78-14c5-4e6e-ace7-893681ac450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:25.120989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd6e78-14c5-4e6e-ace7-893681ac450c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:25.121246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd6e78-14c5-4e6e-ace7-893681ac450c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:25.121527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd6e78-14c5-4e6e-ace7-893681ac450c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:25.122159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd6e78-14c5-4e6e-ace7-893681ac450c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:25.123548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd6e78-14c5-4e6e-ace7-893681ac450c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-22dd6e78-14c5-4e6e-ace7-893681ac450c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:25.133538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:25.134490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:25.135930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 433/879] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Jul 28 03:47:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:25.147251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f188a90-e323-4545-a91e-0fe67d74074a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:25.179027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:25.251418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f188a90-e323-4545-a91e-0fe67d74074a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:25.286968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcc711-fd0d-4914-9e65-673b2ce11587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:25.287920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcc711-fd0d-4914-9e65-673b2ce11587 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:25.288399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcc711-fd0d-4914-9e65-673b2ce11587 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:25.288870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcc711-fd0d-4914-9e65-673b2ce11587 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:25.289903 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcc711-fd0d-4914-9e65-673b2ce11587 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:25.290402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcc711-fd0d-4914-9e65-673b2ce11587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:25.292009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcc711-fd0d-4914-9e65-673b2ce11587 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:25.297379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 447/880] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jul 28 03:47:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:25.303391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd6e78-14c5-4e6e-ace7-893681ac450c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:25.410882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd6e78-14c5-4e6e-ace7-893681ac450c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:25.447467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc89a2-5289-4264-8528-f011166ee29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:25.448094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc89a2-5289-4264-8528-f011166ee29f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:25.448382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc89a2-5289-4264-8528-f011166ee29f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:25.448656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc89a2-5289-4264-8528-f011166ee29f None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:25.449341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc89a2-5289-4264-8528-f011166ee29f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:25.449645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc89a2-5289-4264-8528-f011166ee29f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:25.451030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc89a2-5289-4264-8528-f011166ee29f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:25.453123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:25.457195 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 434/881] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jul 28 03:47:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:25.462902 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcc711-fd0d-4914-9e65-673b2ce11587 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:25.570105 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:25.581346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcc711-fd0d-4914-9e65-673b2ce11587 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:25.617745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32d19f-f7e1-458a-8ac5-7db3ff586c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:25.618360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32d19f-f7e1-458a-8ac5-7db3ff586c18 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:25.618642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32d19f-f7e1-458a-8ac5-7db3ff586c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:25.618946 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32d19f-f7e1-458a-8ac5-7db3ff586c18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:25.619709 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32d19f-f7e1-458a-8ac5-7db3ff586c18 None admin] RBAC: Adding query filter params (interface=public, service_id=be9f42c8e92f4d7ba364e0aeae5493ee, region_id=RegionOne) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:25.620015 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32d19f-f7e1-458a-8ac5-7db3ff586c18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:25.621464 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32d19f-f7e1-458a-8ac5-7db3ff586c18 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:25.627345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 448/882] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jul 28 03:47:25 2022] GET /identity/v3/endpoints?service_id=be9f42c8e92f4d7ba364e0aeae5493ee&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:25.635880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc89a2-5289-4264-8528-f011166ee29f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:25.747559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc89a2-5289-4264-8528-f011166ee29f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:25.766880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249d64-3535-495e-a471-c6fce79ac3b0 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:25.767191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249d64-3535-495e-a471-c6fce79ac3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:25.767446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249d64-3535-495e-a471-c6fce79ac3b0 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:25.769830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 449/883] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:47:25.774902 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a926919-0311-40db-91c1-101d3624f363 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:25.775192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a926919-0311-40db-91c1-101d3624f363 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:25.775472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a926919-0311-40db-91c1-101d3624f363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:25.787309 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c7b71-5fa5-40c4-b284-a9e733c1fe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:25.787930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c7b71-5fa5-40c4-b284-a9e733c1fe31 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:25.788210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c7b71-5fa5-40c4-b284-a9e733c1fe31 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:25.788483 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c7b71-5fa5-40c4-b284-a9e733c1fe31 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:25.789653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c7b71-5fa5-40c4-b284-a9e733c1fe31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:25.791507 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c7b71-5fa5-40c4-b284-a9e733c1fe31 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:25.794876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9a926919-0311-40db-91c1-101d3624f363 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:25.796142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9a926919-0311-40db-91c1-101d3624f363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:25.797619 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 435/884] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jul 28 03:47:25 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:47:25.879828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 450/885] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:25.903005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c7b71-5fa5-40c4-b284-a9e733c1fe31 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:25.906853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde3896-7370-43c1-85fb-34bb1ba1a6f5 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:25.907230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde3896-7370-43c1-85fb-34bb1ba1a6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:25.907517 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde3896-7370-43c1-85fb-34bb1ba1a6f5 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:25.909263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 451/886] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:25.916227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf314702-ddc4-4854-8f4e-2fe2a6d7fd4d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:25.916539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf314702-ddc4-4854-8f4e-2fe2a6d7fd4d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:25.916776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf314702-ddc4-4854-8f4e-2fe2a6d7fd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:25.937831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bf314702-ddc4-4854-8f4e-2fe2a6d7fd4d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:25.939308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bf314702-ddc4-4854-8f4e-2fe2a6d7fd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:25.975978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:26.020776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c7b71-5fa5-40c4-b284-a9e733c1fe31 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:26.025568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 452/887] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:26.049631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf314702-ddc4-4854-8f4e-2fe2a6d7fd4d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:26.064716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd5301-c0c9-4602-be54-b72ca5689b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:26.065389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd5301-c0c9-4602-be54-b72ca5689b0e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.065719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd5301-c0c9-4602-be54-b72ca5689b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.066038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd5301-c0c9-4602-be54-b72ca5689b0e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.066957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd5301-c0c9-4602-be54-b72ca5689b0e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:26.068886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd5301-c0c9-4602-be54-b72ca5689b0e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:26.075046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-4efd5301-c0c9-4602-be54-b72ca5689b0e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:26.082585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:26.083793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:26.084643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 436/888] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:47:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:26.091264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd5301-c0c9-4602-be54-b72ca5689b0e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:26.159847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf314702-ddc4-4854-8f4e-2fe2a6d7fd4d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:26.199748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d72bd2-d1ad-4b44-8165-de78c9d3d7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:26.200394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d72bd2-d1ad-4b44-8165-de78c9d3d7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.200672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d72bd2-d1ad-4b44-8165-de78c9d3d7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.201029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d72bd2-d1ad-4b44-8165-de78c9d3d7b0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.215239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d72bd2-d1ad-4b44-8165-de78c9d3d7b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-20d72bd2-d1ad-4b44-8165-de78c9d3d7b0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:26.222681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:26.223793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:26.224629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 453/889] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jul 28 03:47:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:26.230704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd5301-c0c9-4602-be54-b72ca5689b0e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:26.232282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d72bd2-d1ad-4b44-8165-de78c9d3d7b0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:26.261373 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:26.272548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920f76f-5608-44c9-a7ed-0313c6fddbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:26.273113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920f76f-5608-44c9-a7ed-0313c6fddbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.273394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920f76f-5608-44c9-a7ed-0313c6fddbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.273664 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920f76f-5608-44c9-a7ed-0313c6fddbf8 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.275159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920f76f-5608-44c9-a7ed-0313c6fddbf8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:26.275476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920f76f-5608-44c9-a7ed-0313c6fddbf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:26.281034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920f76f-5608-44c9-a7ed-0313c6fddbf8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:26.291150 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 437/890] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:47:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:26.293650 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920f76f-5608-44c9-a7ed-0313c6fddbf8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:26.334522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:26.360045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d72bd2-d1ad-4b44-8165-de78c9d3d7b0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:26.399612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b2452-66a0-4a13-8908-5e7179488797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:26.400296 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b2452-66a0-4a13-8908-5e7179488797 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.400581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b2452-66a0-4a13-8908-5e7179488797 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.400904 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b2452-66a0-4a13-8908-5e7179488797 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.401742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b2452-66a0-4a13-8908-5e7179488797 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:26.402055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b2452-66a0-4a13-8908-5e7179488797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:26.403568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b2452-66a0-4a13-8908-5e7179488797 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:26.413818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920f76f-5608-44c9-a7ed-0313c6fddbf8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:26.418072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 454/891] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jul 28 03:47:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:26.419002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b2452-66a0-4a13-8908-5e7179488797 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:26.452378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b3987-6198-4e2e-9726-cb5842da4e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:26.453003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b3987-6198-4e2e-9726-cb5842da4e5c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.453281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b3987-6198-4e2e-9726-cb5842da4e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.453642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b3987-6198-4e2e-9726-cb5842da4e5c None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.454312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b3987-6198-4e2e-9726-cb5842da4e5c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:26.454730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b3987-6198-4e2e-9726-cb5842da4e5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:26.456738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b3987-6198-4e2e-9726-cb5842da4e5c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:26.467034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 438/892] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:47:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:26.467927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b3987-6198-4e2e-9726-cb5842da4e5c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:26.530142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:26.538672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b2452-66a0-4a13-8908-5e7179488797 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:26.577596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6a11d-f373-476d-885f-bafd4d9bebe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:26.578291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6a11d-f373-476d-885f-bafd4d9bebe3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.578511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6a11d-f373-476d-885f-bafd4d9bebe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.578797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6a11d-f373-476d-885f-bafd4d9bebe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.586139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b3987-6198-4e2e-9726-cb5842da4e5c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:26.590264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6a11d-f373-476d-885f-bafd4d9bebe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:26.591747 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6a11d-f373-476d-885f-bafd4d9bebe3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:26.593644 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-abf6a11d-f373-476d-885f-bafd4d9bebe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:26.609175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:26.610060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:26.611549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:26.611549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:26.611549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:26.611549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:26.611549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 455/893] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:26.611549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de16b-2aa6-46b6-ad53-e96b563118be None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.611903 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de16b-2aa6-46b6-ad53-e96b563118be None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.612117 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de16b-2aa6-46b6-ad53-e96b563118be None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.614076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 456/894] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:26.617889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5de16b-2aa6-46b6-ad53-e96b563118be None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:26.633085 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efff66-945a-42ac-810c-4e6c1b401c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:26.633704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efff66-945a-42ac-810c-4e6c1b401c40 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.633989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efff66-945a-42ac-810c-4e6c1b401c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.634263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efff66-945a-42ac-810c-4e6c1b401c40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.635048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efff66-945a-42ac-810c-4e6c1b401c40 None admin] RBAC: Adding query filter params (interface=public, service_id=62dce52f4aee477db3c6b55e3953b904, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:26.635360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efff66-945a-42ac-810c-4e6c1b401c40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:26.636718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efff66-945a-42ac-810c-4e6c1b401c40 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:26.641985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 439/895] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jul 28 03:47:26 2022] GET /identity/v3/endpoints?service_id=62dce52f4aee477db3c6b55e3953b904&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:26.643072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fc6e3-6882-453b-ab85-e9688571449e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.643417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fc6e3-6882-453b-ab85-e9688571449e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.643734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fc6e3-6882-453b-ab85-e9688571449e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.661903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bb6fc6e3-6882-453b-ab85-e9688571449e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:26.663123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bb6fc6e3-6882-453b-ab85-e9688571449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:26.730503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5de16b-2aa6-46b6-ad53-e96b563118be None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:26.732284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 440/896] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:26.743672 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fc6e3-6882-453b-ab85-e9688571449e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:26.766321 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e1279-81a7-4803-850e-2c69a0efa514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:26.766966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e1279-81a7-4803-850e-2c69a0efa514 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.767247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e1279-81a7-4803-850e-2c69a0efa514 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.767522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e1279-81a7-4803-850e-2c69a0efa514 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.768221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e1279-81a7-4803-850e-2c69a0efa514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:26.769033 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e1279-81a7-4803-850e-2c69a0efa514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:26.770392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e1279-81a7-4803-850e-2c69a0efa514 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:26.787107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 457/897] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:26.787107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e1279-81a7-4803-850e-2c69a0efa514 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:26.807701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:26.845592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:26.858087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fc6e3-6882-453b-ab85-e9688571449e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:26.895503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e1279-81a7-4803-850e-2c69a0efa514 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:26.897941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6e4a1-416b-4443-94a7-b7bd96ceec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:26.898636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6e4a1-416b-4443-94a7-b7bd96ceec23 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.898952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6e4a1-416b-4443-94a7-b7bd96ceec23 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.899229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6e4a1-416b-4443-94a7-b7bd96ceec23 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.899881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6e4a1-416b-4443-94a7-b7bd96ceec23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:26.901236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6e4a1-416b-4443-94a7-b7bd96ceec23 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:26.907345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 441/898] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jul 28 03:47:26 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:47:26.907935 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6e4a1-416b-4443-94a7-b7bd96ceec23 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:26.957881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cb8ef-54b3-4614-9536-18b804b5a89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:26.959379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cb8ef-54b3-4614-9536-18b804b5a89b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:26.959824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cb8ef-54b3-4614-9536-18b804b5a89b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:26.960702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cb8ef-54b3-4614-9536-18b804b5a89b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:26.961870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cb8ef-54b3-4614-9536-18b804b5a89b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:26.963801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cb8ef-54b3-4614-9536-18b804b5a89b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-2f7cb8ef-54b3-4614-9536-18b804b5a89b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:26.978335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:26.979659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:26.980683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:26.984347 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 458/899] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jul 28 03:47:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:26.992031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cb8ef-54b3-4614-9536-18b804b5a89b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:27.047818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6e4a1-416b-4443-94a7-b7bd96ceec23 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:27.064798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:27.091389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9bd95-dfbd-47b5-8bf3-d2e1e74b062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:27.092402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9bd95-dfbd-47b5-8bf3-d2e1e74b062f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:27.092687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9bd95-dfbd-47b5-8bf3-d2e1e74b062f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:27.092961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9bd95-dfbd-47b5-8bf3-d2e1e74b062f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:27.103767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9bd95-dfbd-47b5-8bf3-d2e1e74b062f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:27.105311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9bd95-dfbd-47b5-8bf3-d2e1e74b062f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:27.115736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cb8ef-54b3-4614-9536-18b804b5a89b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-cfc9bd95-dfbd-47b5-8bf3-d2e1e74b062f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:27.118052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:27.119062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:27.119980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:27.119980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:27.119980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:27.119980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:27.119980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:27.119980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:27.119980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:27.119980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:27.119980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:27.119980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:27.124017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 442/900] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jul 28 03:47:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:27.125241 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9bd95-dfbd-47b5-8bf3-d2e1e74b062f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:27.137388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:27.155674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2863bbe-2406-49e4-a11b-f36b113a9272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:27.156301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2863bbe-2406-49e4-a11b-f36b113a9272 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:27.156631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2863bbe-2406-49e4-a11b-f36b113a9272 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:27.156910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2863bbe-2406-49e4-a11b-f36b113a9272 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:27.157592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2863bbe-2406-49e4-a11b-f36b113a9272 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:27.157900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2863bbe-2406-49e4-a11b-f36b113a9272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:27.159963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2863bbe-2406-49e4-a11b-f36b113a9272 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:27.170825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 459/901] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jul 28 03:47:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:27.173620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2863bbe-2406-49e4-a11b-f36b113a9272 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:27.245986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9bd95-dfbd-47b5-8bf3-d2e1e74b062f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:27.287656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073506e-9625-4dfb-8d41-ba5b06f6f1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:27.289112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073506e-9625-4dfb-8d41-ba5b06f6f1fb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:27.289583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073506e-9625-4dfb-8d41-ba5b06f6f1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:27.290040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073506e-9625-4dfb-8d41-ba5b06f6f1fb None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:27.291517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073506e-9625-4dfb-8d41-ba5b06f6f1fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:27.292431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073506e-9625-4dfb-8d41-ba5b06f6f1fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:27.294343 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073506e-9625-4dfb-8d41-ba5b06f6f1fb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:27.316768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2863bbe-2406-49e4-a11b-f36b113a9272 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:27.321211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 443/902] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jul 28 03:47:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:27.328043 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073506e-9625-4dfb-8d41-ba5b06f6f1fb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:27.357895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1235284-6dbc-4409-b9d4-172233eb8b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:27.358555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1235284-6dbc-4409-b9d4-172233eb8b7c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:27.358884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1235284-6dbc-4409-b9d4-172233eb8b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:27.359167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1235284-6dbc-4409-b9d4-172233eb8b7c None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:27.359856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1235284-6dbc-4409-b9d4-172233eb8b7c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:27.360183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1235284-6dbc-4409-b9d4-172233eb8b7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:27.361524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1235284-6dbc-4409-b9d4-172233eb8b7c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:27.370848 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 460/903] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jul 28 03:47:27 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:27.379795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:27.461167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073506e-9625-4dfb-8d41-ba5b06f6f1fb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:27.498459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911aafd-cafa-4f9f-8dab-1b6c965c1d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:27.499146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911aafd-cafa-4f9f-8dab-1b6c965c1d6f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:27.499428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911aafd-cafa-4f9f-8dab-1b6c965c1d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:27.499700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911aafd-cafa-4f9f-8dab-1b6c965c1d6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:27.582015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911aafd-cafa-4f9f-8dab-1b6c965c1d6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:27.583455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911aafd-cafa-4f9f-8dab-1b6c965c1d6f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:27.616132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:27.694696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 444/904] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:27 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 03:47:27.699981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1235284-6dbc-4409-b9d4-172233eb8b7c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:27.750454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:27.809046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1235284-6dbc-4409-b9d4-172233eb8b7c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:27.845407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12876ae6-221f-496d-9bf1-c9095666cdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:27.846297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12876ae6-221f-496d-9bf1-c9095666cdea None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:27.847435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12876ae6-221f-496d-9bf1-c9095666cdea None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:27.847935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12876ae6-221f-496d-9bf1-c9095666cdea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:27.859251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12876ae6-221f-496d-9bf1-c9095666cdea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:27.860909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12876ae6-221f-496d-9bf1-c9095666cdea None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-12876ae6-221f-496d-9bf1-c9095666cdea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:27.876028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:27.877157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:27.878886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:27.878886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:27.878886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:27.878886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:27.879141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c221cfc-3987-4064-9e15-71dc8bfed67e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:27.879141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c221cfc-3987-4064-9e15-71dc8bfed67e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:27.879141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c221cfc-3987-4064-9e15-71dc8bfed67e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:27.879141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 445/905] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:27.880980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 461/906] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:27.885460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cde53-6a68-45b5-a42a-6247085a2bab None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:27.885460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cde53-6a68-45b5-a42a-6247085a2bab None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:27.885460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cde53-6a68-45b5-a42a-6247085a2bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:27.886672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12876ae6-221f-496d-9bf1-c9095666cdea None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:27.903963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8f6cde53-6a68-45b5-a42a-6247085a2bab None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:27.905039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8f6cde53-6a68-45b5-a42a-6247085a2bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:27.989196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 446/907] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:28.000354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:28.005237 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12876ae6-221f-496d-9bf1-c9095666cdea None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:28.015603 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cde53-6a68-45b5-a42a-6247085a2bab None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:28.044910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206045dd-84fd-4540-b93a-e2915e7c8531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:28.045632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206045dd-84fd-4540-b93a-e2915e7c8531 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:28.045925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206045dd-84fd-4540-b93a-e2915e7c8531 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:28.046201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206045dd-84fd-4540-b93a-e2915e7c8531 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:28.046924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206045dd-84fd-4540-b93a-e2915e7c8531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:28.047342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206045dd-84fd-4540-b93a-e2915e7c8531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:28.048717 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206045dd-84fd-4540-b93a-e2915e7c8531 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:28.062439 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 462/908] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:28.068345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206045dd-84fd-4540-b93a-e2915e7c8531 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:28.089614 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:28.134911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cde53-6a68-45b5-a42a-6247085a2bab None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:28.171610 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be8e82-edc3-4d9e-aa71-a5e9cc6ca5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:28.172261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be8e82-edc3-4d9e-aa71-a5e9cc6ca5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:28.172598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be8e82-edc3-4d9e-aa71-a5e9cc6ca5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:28.172877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be8e82-edc3-4d9e-aa71-a5e9cc6ca5f7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:28.173510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be8e82-edc3-4d9e-aa71-a5e9cc6ca5f7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:28.175806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be8e82-edc3-4d9e-aa71-a5e9cc6ca5f7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d6be8e82-edc3-4d9e-aa71-a5e9cc6ca5f7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:28.185468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:28.186415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:28.187271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 447/909] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Jul 28 03:47:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:28.192196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206045dd-84fd-4540-b93a-e2915e7c8531 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:28.194459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be8e82-edc3-4d9e-aa71-a5e9cc6ca5f7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:28.234749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c078fa-9043-4bc1-aa70-ede3b5223795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:28.235773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c078fa-9043-4bc1-aa70-ede3b5223795 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:28.236365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c078fa-9043-4bc1-aa70-ede3b5223795 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:28.236842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c078fa-9043-4bc1-aa70-ede3b5223795 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:28.245829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c078fa-9043-4bc1-aa70-ede3b5223795 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:28.247713 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c078fa-9043-4bc1-aa70-ede3b5223795 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-c1c078fa-9043-4bc1-aa70-ede3b5223795 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:28.261147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:28.262050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:28.262974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:28.262974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:28.262974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:28.262974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:28.262974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:28.262974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:28.262974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:28.262974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:28.265588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 463/910] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:28.271109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c078fa-9043-4bc1-aa70-ede3b5223795 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:28.300805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:28.311897 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be8e82-edc3-4d9e-aa71-a5e9cc6ca5f7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:28.348773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:28.358476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23a0d9-8b80-449a-833c-56f525c9c320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:28.359591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23a0d9-8b80-449a-833c-56f525c9c320 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:28.359591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23a0d9-8b80-449a-833c-56f525c9c320 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:28.359795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23a0d9-8b80-449a-833c-56f525c9c320 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:28.360439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23a0d9-8b80-449a-833c-56f525c9c320 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:28.360748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23a0d9-8b80-449a-833c-56f525c9c320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:28.362114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23a0d9-8b80-449a-833c-56f525c9c320 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:28.372729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 448/911] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jul 28 03:47:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:28.374056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23a0d9-8b80-449a-833c-56f525c9c320 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:28.415075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c078fa-9043-4bc1-aa70-ede3b5223795 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:28.461947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5590bc6-9a16-4ba1-b73f-d7066554c2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:28.462917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5590bc6-9a16-4ba1-b73f-d7066554c2ef None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:28.463339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5590bc6-9a16-4ba1-b73f-d7066554c2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:28.463631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5590bc6-9a16-4ba1-b73f-d7066554c2ef None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:28.464538 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5590bc6-9a16-4ba1-b73f-d7066554c2ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:28.464852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5590bc6-9a16-4ba1-b73f-d7066554c2ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:28.466473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5590bc6-9a16-4ba1-b73f-d7066554c2ef None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:28.488253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 464/912] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:28.489910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5590bc6-9a16-4ba1-b73f-d7066554c2ef None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:28.493007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23a0d9-8b80-449a-833c-56f525c9c320 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:28.528394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6d392-cfea-457e-8c5b-5426143e16d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:28.529019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6d392-cfea-457e-8c5b-5426143e16d5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:28.529304 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6d392-cfea-457e-8c5b-5426143e16d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:28.529582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6d392-cfea-457e-8c5b-5426143e16d5 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:28.530263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6d392-cfea-457e-8c5b-5426143e16d5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:28.530568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6d392-cfea-457e-8c5b-5426143e16d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:28.532198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6d392-cfea-457e-8c5b-5426143e16d5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:28.542627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 449/913] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jul 28 03:47:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:28.546614 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6d392-cfea-457e-8c5b-5426143e16d5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:28.599857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:28.618256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5590bc6-9a16-4ba1-b73f-d7066554c2ef None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:28.620040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:28.669294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6d392-cfea-457e-8c5b-5426143e16d5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:28.674573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509314ea-a702-4e7e-970e-53346bc4da3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:28.675702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509314ea-a702-4e7e-970e-53346bc4da3f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:28.676197 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509314ea-a702-4e7e-970e-53346bc4da3f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:28.677217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509314ea-a702-4e7e-970e-53346bc4da3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:28.678250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509314ea-a702-4e7e-970e-53346bc4da3f None admin] RBAC: Adding query filter params (role.id=daec222fcf874e5fa9f48fc8192ea9a2, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=3b23317779d94f99bf804a172bb9c4ed) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:28.679236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509314ea-a702-4e7e-970e-53346bc4da3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:28.680694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509314ea-a702-4e7e-970e-53346bc4da3f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:28.694810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 465/914] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:28 2022] GET /identity/v3/role_assignments?user.id=3b23317779d94f99bf804a172bb9c4ed&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=daec222fcf874e5fa9f48fc8192ea9a2 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:28.718553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c275bcf-5d18-481d-b28e-1ebf84e80c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:28.719213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c275bcf-5d18-481d-b28e-1ebf84e80c02 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:28.719493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c275bcf-5d18-481d-b28e-1ebf84e80c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:28.719764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c275bcf-5d18-481d-b28e-1ebf84e80c02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:28.720484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c275bcf-5d18-481d-b28e-1ebf84e80c02 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:28.722469 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c275bcf-5d18-481d-b28e-1ebf84e80c02 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:28.746087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 450/915] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:47:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:29.011282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac9de3-c2fd-440d-90ca-f50d80865ebd None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:29.011772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac9de3-c2fd-440d-90ca-f50d80865ebd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:29.011849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac9de3-c2fd-440d-90ca-f50d80865ebd None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:29.013452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 466/916] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:29.019405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7755a3-e6e3-4c76-b1b5-3f79d1cd9493 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:29.019696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7755a3-e6e3-4c76-b1b5-3f79d1cd9493 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:29.019973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7755a3-e6e3-4c76-b1b5-3f79d1cd9493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:29.039410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1e7755a3-e6e3-4c76-b1b5-3f79d1cd9493 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:29.040605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1e7755a3-e6e3-4c76-b1b5-3f79d1cd9493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:29.115911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 451/917] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:29.137954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac9de3-c2fd-440d-90ca-f50d80865ebd None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:29.257404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac9de3-c2fd-440d-90ca-f50d80865ebd None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:29.293821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f9bfb-2f0f-420c-bba6-6cc15ceec97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:29.294390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f9bfb-2f0f-420c-bba6-6cc15ceec97e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:29.294723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f9bfb-2f0f-420c-bba6-6cc15ceec97e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:29.295040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f9bfb-2f0f-420c-bba6-6cc15ceec97e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:29.295722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f9bfb-2f0f-420c-bba6-6cc15ceec97e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:29.297090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f9bfb-2f0f-420c-bba6-6cc15ceec97e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-544f9bfb-2f0f-420c-bba6-6cc15ceec97e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:29.306801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:29.308045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 03:47:29.309100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:29.310893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 467/918] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:47:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:29.314338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7755a3-e6e3-4c76-b1b5-3f79d1cd9493 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:29.359255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:29.419830 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7755a3-e6e3-4c76-b1b5-3f79d1cd9493 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:29.455155 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36735c-8964-433a-bde3-46015cb47be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:29.455777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36735c-8964-433a-bde3-46015cb47be0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:29.456812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36735c-8964-433a-bde3-46015cb47be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:29.457093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36735c-8964-433a-bde3-46015cb47be0 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:29.457784 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36735c-8964-433a-bde3-46015cb47be0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:29.458085 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36735c-8964-433a-bde3-46015cb47be0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:29.459473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36735c-8964-433a-bde3-46015cb47be0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:29.464691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 452/919] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:47:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:29.470243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f9bfb-2f0f-420c-bba6-6cc15ceec97e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:29.481366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:29.582651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f9bfb-2f0f-420c-bba6-6cc15ceec97e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:29.587789 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9360f34-5aac-45da-a0d8-35ee00b1e8bd None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:29.588198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9360f34-5aac-45da-a0d8-35ee00b1e8bd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:29.588549 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9360f34-5aac-45da-a0d8-35ee00b1e8bd None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:29.591218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 453/920] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:47:29.595482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccd4ff-02c1-4442-b493-164dcf0ada33 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:29.595690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccd4ff-02c1-4442-b493-164dcf0ada33 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:29.595967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccd4ff-02c1-4442-b493-164dcf0ada33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:29.614402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-53ccd4ff-02c1-4442-b493-164dcf0ada33 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:29.615633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-53ccd4ff-02c1-4442-b493-164dcf0ada33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:29.620425 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a42d11-3f9c-4268-a40b-23455359c44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:29.621250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a42d11-3f9c-4268-a40b-23455359c44b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:29.621713 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a42d11-3f9c-4268-a40b-23455359c44b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:29.622189 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a42d11-3f9c-4268-a40b-23455359c44b None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:29.623188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a42d11-3f9c-4268-a40b-23455359c44b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:29.623675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a42d11-3f9c-4268-a40b-23455359c44b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:29.625330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a42d11-3f9c-4268-a40b-23455359c44b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:29.631308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 468/921] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jul 28 03:47:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:29.642834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a42d11-3f9c-4268-a40b-23455359c44b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:29.699233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 454/922] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:29.711324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:29.719429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccd4ff-02c1-4442-b493-164dcf0ada33 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:29.743514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a42d11-3f9c-4268-a40b-23455359c44b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:29.753220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:29.778525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f96b4e-74da-4ebb-8235-40cbef972afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:29.779191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f96b4e-74da-4ebb-8235-40cbef972afa None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:29.779473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f96b4e-74da-4ebb-8235-40cbef972afa None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:29.779746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f96b4e-74da-4ebb-8235-40cbef972afa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:29.780461 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f96b4e-74da-4ebb-8235-40cbef972afa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:29.781810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f96b4e-74da-4ebb-8235-40cbef972afa None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:29.810265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 469/923] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:47:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:29.840164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccd4ff-02c1-4442-b493-164dcf0ada33 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:29.887883 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cc545-dc7d-41c4-9456-c8f9f7ba4595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:29.888512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cc545-dc7d-41c4-9456-c8f9f7ba4595 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:29.888790 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cc545-dc7d-41c4-9456-c8f9f7ba4595 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:29.889062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cc545-dc7d-41c4-9456-c8f9f7ba4595 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:29.889950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cc545-dc7d-41c4-9456-c8f9f7ba4595 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:29.891733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cc545-dc7d-41c4-9456-c8f9f7ba4595 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:29.911629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 455/924] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:47:29 2022] POST /identity/v3/services => generated 241 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:30.832000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580827d0-d4f1-4cec-ac95-204a98bd147b None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:30.832628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580827d0-d4f1-4cec-ac95-204a98bd147b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:30.832628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580827d0-d4f1-4cec-ac95-204a98bd147b None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:30.834165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 470/925] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:30.835001 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad9060-bbd5-4513-891b-1216aca63bac None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:30.835467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad9060-bbd5-4513-891b-1216aca63bac None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:30.835552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad9060-bbd5-4513-891b-1216aca63bac None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:30.837103 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 456/926] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:30.838928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f05c35-a899-4b2b-a402-e7848aa85e0c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:30.839223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f05c35-a899-4b2b-a402-e7848aa85e0c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:30.839506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f05c35-a899-4b2b-a402-e7848aa85e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:30.843261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68a33a-72a1-4eb3-8e56-cc3036bf6746 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:30.843551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68a33a-72a1-4eb3-8e56-cc3036bf6746 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:30.843828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68a33a-72a1-4eb3-8e56-cc3036bf6746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:30.859108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-25f05c35-a899-4b2b-a402-e7848aa85e0c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:30.860299 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-25f05c35-a899-4b2b-a402-e7848aa85e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:30.862401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0d68a33a-72a1-4eb3-8e56-cc3036bf6746 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:30.863754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0d68a33a-72a1-4eb3-8e56-cc3036bf6746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:30.947586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 457/928] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:30.950861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 471/928] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:30.969179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68a33a-72a1-4eb3-8e56-cc3036bf6746 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:30.970547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f05c35-a899-4b2b-a402-e7848aa85e0c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:30.989122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:31.032429 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:31.095930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f05c35-a899-4b2b-a402-e7848aa85e0c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:31.096136 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68a33a-72a1-4eb3-8e56-cc3036bf6746 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:31.133426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8672556-1edd-4115-aab2-bb5679e003c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:31.134177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8672556-1edd-4115-aab2-bb5679e003c1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:31.134537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8672556-1edd-4115-aab2-bb5679e003c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:31.134918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8672556-1edd-4115-aab2-bb5679e003c1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:31.135947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1001c-f705-4ba9-8cc2-a8a6f25fb061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:31.136772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1001c-f705-4ba9-8cc2-a8a6f25fb061 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:31.137128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1001c-f705-4ba9-8cc2-a8a6f25fb061 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:31.137503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1001c-f705-4ba9-8cc2-a8a6f25fb061 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:31.138262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1001c-f705-4ba9-8cc2-a8a6f25fb061 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:31.139740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1001c-f705-4ba9-8cc2-a8a6f25fb061 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:31.145593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8672556-1edd-4115-aab2-bb5679e003c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-0ee1001c-f705-4ba9-8cc2-a8a6f25fb061 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:31.149422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:31.150484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 03:47:31.151443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b8672556-1edd-4115-aab2-bb5679e003c1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:31.152229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:31.153096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:31.153942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:31.154613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 472/929] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:31.154689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 458/930] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jul 28 03:47:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:31.160327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1001c-f705-4ba9-8cc2-a8a6f25fb061 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:31.160550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8672556-1edd-4115-aab2-bb5679e003c1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:31.242479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:31.268041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8672556-1edd-4115-aab2-bb5679e003c1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:31.276943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1001c-f705-4ba9-8cc2-a8a6f25fb061 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:31.294564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:31.305464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7dc154-7866-44d3-8bf1-28ccd78d242f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:31.306113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dc154-7866-44d3-8bf1-28ccd78d242f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:31.306470 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dc154-7866-44d3-8bf1-28ccd78d242f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:31.306751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dc154-7866-44d3-8bf1-28ccd78d242f None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:31.307564 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7dc154-7866-44d3-8bf1-28ccd78d242f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:31.307877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7dc154-7866-44d3-8bf1-28ccd78d242f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:31.309345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7dc154-7866-44d3-8bf1-28ccd78d242f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:31.314966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643de614-3491-4057-990d-18d3c385c2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:31.315918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643de614-3491-4057-990d-18d3c385c2ba None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:31.316199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643de614-3491-4057-990d-18d3c385c2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:31.316470 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643de614-3491-4057-990d-18d3c385c2ba None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:31.317182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643de614-3491-4057-990d-18d3c385c2ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:31.317484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643de614-3491-4057-990d-18d3c385c2ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:31.318929 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643de614-3491-4057-990d-18d3c385c2ba None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:31.319322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 459/931] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:31.320283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7dc154-7866-44d3-8bf1-28ccd78d242f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:31.332737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 473/932] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jul 28 03:47:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:31.334067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643de614-3491-4057-990d-18d3c385c2ba None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:31.429065 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7dc154-7866-44d3-8bf1-28ccd78d242f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:31.439484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643de614-3491-4057-990d-18d3c385c2ba None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:31.467259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8b546-2fab-4bd6-9ade-348365ee3519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:31.467886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8b546-2fab-4bd6-9ade-348365ee3519 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:31.468270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8b546-2fab-4bd6-9ade-348365ee3519 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:31.468556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8b546-2fab-4bd6-9ade-348365ee3519 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:31.469349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8b546-2fab-4bd6-9ade-348365ee3519 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:31.469661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8b546-2fab-4bd6-9ade-348365ee3519 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:31.471176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8b546-2fab-4bd6-9ade-348365ee3519 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:31.477818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c2301-864f-4845-8783-109bb83e26fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:31.478457 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c2301-864f-4845-8783-109bb83e26fe None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:31.478734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c2301-864f-4845-8783-109bb83e26fe None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:31.479039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c2301-864f-4845-8783-109bb83e26fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:31.482232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 460/933] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:31.491419 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c2301-864f-4845-8783-109bb83e26fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:31.492919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c2301-864f-4845-8783-109bb83e26fe None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-312c2301-864f-4845-8783-109bb83e26fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:31.509754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:31.512012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:31.514155 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:31.514155 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:31.514155 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:31.514155 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:31.517702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 474/934] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:31.517956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8b546-2fab-4bd6-9ade-348365ee3519 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:31.521506 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:31.645664 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8b546-2fab-4bd6-9ade-348365ee3519 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:31.683293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd4123-a9eb-4d66-8799-e6126fe80251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:31.683828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd4123-a9eb-4d66-8799-e6126fe80251 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:31.684107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd4123-a9eb-4d66-8799-e6126fe80251 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:31.684379 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd4123-a9eb-4d66-8799-e6126fe80251 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:31.685131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd4123-a9eb-4d66-8799-e6126fe80251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:31.685432 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd4123-a9eb-4d66-8799-e6126fe80251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:31.686852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd4123-a9eb-4d66-8799-e6126fe80251 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:31.702923 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 461/935] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:31.705106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c2301-864f-4845-8783-109bb83e26fe None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:31.737912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:31.809508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c2301-864f-4845-8783-109bb83e26fe None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:31.844188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4653ab8-a64c-42d7-871a-15704c6cf455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:31.844833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4653ab8-a64c-42d7-871a-15704c6cf455 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:31.845084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4653ab8-a64c-42d7-871a-15704c6cf455 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:31.845363 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4653ab8-a64c-42d7-871a-15704c6cf455 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:31.853076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4653ab8-a64c-42d7-871a-15704c6cf455 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:31.854504 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4653ab8-a64c-42d7-871a-15704c6cf455 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d4653ab8-a64c-42d7-871a-15704c6cf455 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:31.866487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:31.867540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:31.868493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:31.868493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:31.868493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:31.868493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:31.868493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:31.868493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:31.868493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:31.868493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:31.868493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:31.868493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:31.870918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 475/936] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jul 28 03:47:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:31.873831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd4123-a9eb-4d66-8799-e6126fe80251 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:31.942726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:31.971823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8814d-4949-4531-bb03-045af2472ffb None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:31.972376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8814d-4949-4531-bb03-045af2472ffb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:31.972656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8814d-4949-4531-bb03-045af2472ffb None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:31.974806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 476/937] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:47:31.983316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c09727-dd4c-4042-8828-395c3da108be None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:31.983605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c09727-dd4c-4042-8828-395c3da108be None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:31.983872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd4123-a9eb-4d66-8799-e6126fe80251 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:31.983991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c09727-dd4c-4042-8828-395c3da108be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.003991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a9c09727-dd4c-4042-8828-395c3da108be None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:32.005294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a9c09727-dd4c-4042-8828-395c3da108be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:32.030579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9651e1-6c09-4038-b8e5-0206de40feeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:32.031245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9651e1-6c09-4038-b8e5-0206de40feeb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.031529 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9651e1-6c09-4038-b8e5-0206de40feeb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.031806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9651e1-6c09-4038-b8e5-0206de40feeb None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.032608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9651e1-6c09-4038-b8e5-0206de40feeb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:32.032919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9651e1-6c09-4038-b8e5-0206de40feeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:32.034392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9651e1-6c09-4038-b8e5-0206de40feeb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:32.053739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 462/938] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jul 28 03:47:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:32.059765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9651e1-6c09-4038-b8e5-0206de40feeb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:32.084860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 477/939] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:32.086120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54077a0e-1357-40d3-93a0-7cb072ee47fb None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.086853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54077a0e-1357-40d3-93a0-7cb072ee47fb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.086853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54077a0e-1357-40d3-93a0-7cb072ee47fb None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.088001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 478/940] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:32.092985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf5706-9f37-468b-b221-71041cc45a04 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.093271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf5706-9f37-468b-b221-71041cc45a04 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.093553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf5706-9f37-468b-b221-71041cc45a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.118425 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bcbf5706-9f37-468b-b221-71041cc45a04 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:32.119724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bcbf5706-9f37-468b-b221-71041cc45a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:32.135260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:32.174850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9651e1-6c09-4038-b8e5-0206de40feeb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:32.198922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 479/941] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:32.198922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf5706-9f37-468b-b221-71041cc45a04 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:32.202260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:32.218009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93c24a-1bf1-4d5f-88a9-46f83a7b2b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:32.218656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93c24a-1bf1-4d5f-88a9-46f83a7b2b8b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.218967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93c24a-1bf1-4d5f-88a9-46f83a7b2b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.219239 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93c24a-1bf1-4d5f-88a9-46f83a7b2b8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.230543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93c24a-1bf1-4d5f-88a9-46f83a7b2b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:32.232017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93c24a-1bf1-4d5f-88a9-46f83a7b2b8b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-cb93c24a-1bf1-4d5f-88a9-46f83a7b2b8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:32.247349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:32.248301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:32.249211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:32.249211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:32.249211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:32.249211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:32.253972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 463/942] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:32.254582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93c24a-1bf1-4d5f-88a9-46f83a7b2b8b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:32.309952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf5706-9f37-468b-b221-71041cc45a04 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:32.346311 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4411577-c7f5-4fdb-b56d-2d968e001343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:32.347079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411577-c7f5-4fdb-b56d-2d968e001343 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.347285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411577-c7f5-4fdb-b56d-2d968e001343 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.347535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411577-c7f5-4fdb-b56d-2d968e001343 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.348217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4411577-c7f5-4fdb-b56d-2d968e001343 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:32.349646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4411577-c7f5-4fdb-b56d-2d968e001343 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:32.352263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: WARNING keystone.server.flask.common [None req-d4411577-c7f5-4fdb-b56d-2d968e001343 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 03:47:32.364997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93c24a-1bf1-4d5f-88a9-46f83a7b2b8b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:32.371009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.sql.core [None req-d4411577-c7f5-4fdb-b56d-2d968e001343 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 03:47:32.371009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 03:47:32.371009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [parameters: {'id': 'ab1c4110449b47608c83abba7d6d48d9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 03:47:32.371009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80473) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d4411577-c7f5-4fdb-b56d-2d968e001343 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ab1c4110449b47608c83abba7d6d48d9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ab1c4110449b47608c83abba7d6d48d9. Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application conn.query(q) Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 03:47:32.388233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application result.read() Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 03:47:32.389075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application next(self.gen) Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application yield resource Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application next(self.gen) Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise self.value Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application session.commit() Jul 28 03:47:32.390025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self.session.flush() Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application compat.raise_( Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 03:47:32.391337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 03:47:32.392218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application conn.query(q) Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application result.read() Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 03:47:32.392994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [parameters: {'id': 'ab1c4110449b47608c83abba7d6d48d9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:32.395618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:32.397318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:32.399248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:32.399248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 03:47:32.399248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 03:47:32.399248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:32.399248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:32.399248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 03:47:32.399248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 03:47:32.399248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ab1c4110449b47608c83abba7d6d48d9. Jul 28 03:47:32.399248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:32.399248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 480/943] 10.222.0.46 () {60 vars in 1140 bytes} [Thu Jul 28 03:47:32 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:32.399248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4411577-c7f5-4fdb-b56d-2d968e001343 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:32.405845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:32.413135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab129c-c008-498d-850c-b1e772283439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:32.413795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab129c-c008-498d-850c-b1e772283439 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.414138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab129c-c008-498d-850c-b1e772283439 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.414342 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab129c-c008-498d-850c-b1e772283439 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.415086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab129c-c008-498d-850c-b1e772283439 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:32.416444 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab129c-c008-498d-850c-b1e772283439 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f0ab129c-c008-498d-850c-b1e772283439 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:32.425661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:32.426690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:32.427553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 464/944] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jul 28 03:47:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:32.428915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab129c-c008-498d-850c-b1e772283439 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:32.471945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:32.510499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4411577-c7f5-4fdb-b56d-2d968e001343 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:32.543760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab129c-c008-498d-850c-b1e772283439 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:32.544818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dba619-5b5c-4682-b0d5-3cf67d7b3820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:32.545500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dba619-5b5c-4682-b0d5-3cf67d7b3820 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.545788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dba619-5b5c-4682-b0d5-3cf67d7b3820 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.546066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dba619-5b5c-4682-b0d5-3cf67d7b3820 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.546754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dba619-5b5c-4682-b0d5-3cf67d7b3820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:32.547088 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dba619-5b5c-4682-b0d5-3cf67d7b3820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:32.548649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dba619-5b5c-4682-b0d5-3cf67d7b3820 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:32.567421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 481/945] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:32.572376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dba619-5b5c-4682-b0d5-3cf67d7b3820 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:32.582197 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa22f59-8791-459b-a743-0f46cc519d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:32.582869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa22f59-8791-459b-a743-0f46cc519d8d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.583157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa22f59-8791-459b-a743-0f46cc519d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.583432 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa22f59-8791-459b-a743-0f46cc519d8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.590213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa22f59-8791-459b-a743-0f46cc519d8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:32.591683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa22f59-8791-459b-a743-0f46cc519d8d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-4fa22f59-8791-459b-a743-0f46cc519d8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:32.603944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:32.604855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:32.605770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:32.605770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:32.605770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:32.605770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:32.605770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:32.605770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:32.605770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:32.605770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:32.605770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 465/946] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:32.606272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa22f59-8791-459b-a743-0f46cc519d8d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:32.651840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:32.688797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dba619-5b5c-4682-b0d5-3cf67d7b3820 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:32.715847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:32.720597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa22f59-8791-459b-a743-0f46cc519d8d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:32.726174 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9492a2f-8995-4ed7-a080-83e0cd6536fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:32.726734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9492a2f-8995-4ed7-a080-83e0cd6536fa None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.727044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9492a2f-8995-4ed7-a080-83e0cd6536fa None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.727318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9492a2f-8995-4ed7-a080-83e0cd6536fa None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.728025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9492a2f-8995-4ed7-a080-83e0cd6536fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:32.728329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9492a2f-8995-4ed7-a080-83e0cd6536fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:32.729682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9492a2f-8995-4ed7-a080-83e0cd6536fa None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:32.734695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 482/947] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jul 28 03:47:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:32.739916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9492a2f-8995-4ed7-a080-83e0cd6536fa None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:32.757973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19603598-90ea-4c62-86ef-6f299b78fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:32.758613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19603598-90ea-4c62-86ef-6f299b78fa1d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.758926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19603598-90ea-4c62-86ef-6f299b78fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.759388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19603598-90ea-4c62-86ef-6f299b78fa1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.766036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19603598-90ea-4c62-86ef-6f299b78fa1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:32.767507 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19603598-90ea-4c62-86ef-6f299b78fa1d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-19603598-90ea-4c62-86ef-6f299b78fa1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:32.779561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:32.780474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:32.781403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:32.781403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:32.781403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:32.781403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:32.781403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:32.781403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:32.781403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:32.781403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:32.786064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 466/948] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:32.787079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19603598-90ea-4c62-86ef-6f299b78fa1d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:32.844723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9492a2f-8995-4ed7-a080-83e0cd6536fa None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:32.876012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:32.883347 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63521d39-89af-4746-b8fa-932172671475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:32.883989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63521d39-89af-4746-b8fa-932172671475 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.884565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63521d39-89af-4746-b8fa-932172671475 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.884853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63521d39-89af-4746-b8fa-932172671475 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.885741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63521d39-89af-4746-b8fa-932172671475 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:32.886057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63521d39-89af-4746-b8fa-932172671475 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:32.888122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63521d39-89af-4746-b8fa-932172671475 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:32.896297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19603598-90ea-4c62-86ef-6f299b78fa1d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:32.908555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 483/949] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:32 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:32.910460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63521d39-89af-4746-b8fa-932172671475 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:32.945055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed906df-3267-4d7f-9b4a-fd94aaa90c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:32.945698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed906df-3267-4d7f-9b4a-fd94aaa90c1b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:32.945979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed906df-3267-4d7f-9b4a-fd94aaa90c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:32.946252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed906df-3267-4d7f-9b4a-fd94aaa90c1b None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:32.946995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed906df-3267-4d7f-9b4a-fd94aaa90c1b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:32.947290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed906df-3267-4d7f-9b4a-fd94aaa90c1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:32.948828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed906df-3267-4d7f-9b4a-fd94aaa90c1b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:32.960732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 467/950] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jul 28 03:47:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:32.962377 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed906df-3267-4d7f-9b4a-fd94aaa90c1b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:32.993975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:33.045046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63521d39-89af-4746-b8fa-932172671475 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:33.094700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d7976-ef95-41a3-a55d-c742a710e0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:33.095678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d7976-ef95-41a3-a55d-c742a710e0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:33.095958 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d7976-ef95-41a3-a55d-c742a710e0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:33.096285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d7976-ef95-41a3-a55d-c742a710e0a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:33.097513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d7976-ef95-41a3-a55d-c742a710e0a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:33.097818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d7976-ef95-41a3-a55d-c742a710e0a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:33.099313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d7976-ef95-41a3-a55d-c742a710e0a1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:33.106886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed906df-3267-4d7f-9b4a-fd94aaa90c1b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:33.114947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 484/951] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:33.120540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d7976-ef95-41a3-a55d-c742a710e0a1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:33.153189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4727a7d-b93b-46c5-ac66-eee8e1163472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:33.153819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4727a7d-b93b-46c5-ac66-eee8e1163472 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:33.154276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4727a7d-b93b-46c5-ac66-eee8e1163472 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:33.154557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4727a7d-b93b-46c5-ac66-eee8e1163472 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:33.155354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4727a7d-b93b-46c5-ac66-eee8e1163472 None admin] RBAC: Adding query filter params (interface=public, service_id=2cea1d4c39ef42cc87461a20e7cd3086, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:33.155663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4727a7d-b93b-46c5-ac66-eee8e1163472 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:33.157021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4727a7d-b93b-46c5-ac66-eee8e1163472 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:33.164065 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 468/952] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jul 28 03:47:32 2022] GET /identity/v3/endpoints?service_id=2cea1d4c39ef42cc87461a20e7cd3086&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:33.169727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4727a7d-b93b-46c5-ac66-eee8e1163472 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:33.182511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:33.252153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d7976-ef95-41a3-a55d-c742a710e0a1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:33.289601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe697d8-835a-427e-8895-6cc992d4f017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:33.290126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:33.290976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe697d8-835a-427e-8895-6cc992d4f017 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:33.291314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe697d8-835a-427e-8895-6cc992d4f017 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:33.291695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe697d8-835a-427e-8895-6cc992d4f017 None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/3b23317779d94f99bf804a172bb9c4ed/roles/daec222fcf874e5fa9f48fc8192ea9a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:33.294978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe697d8-835a-427e-8895-6cc992d4f017 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=3b23317779d94f99bf804a172bb9c4ed, role_id=daec222fcf874e5fa9f48fc8192ea9a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:33.297051 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe697d8-835a-427e-8895-6cc992d4f017 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:33.310680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4727a7d-b93b-46c5-ac66-eee8e1163472 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:33.318967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 485/953] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:47:33 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/3b23317779d94f99bf804a172bb9c4ed/roles/daec222fcf874e5fa9f48fc8192ea9a2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 03:47:33.353503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010a348-f910-48c4-bde0-49848a3b7d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:33.354121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010a348-f910-48c4-bde0-49848a3b7d90 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:33.354405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010a348-f910-48c4-bde0-49848a3b7d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:33.354734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010a348-f910-48c4-bde0-49848a3b7d90 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:33.355434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010a348-f910-48c4-bde0-49848a3b7d90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:33.356820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010a348-f910-48c4-bde0-49848a3b7d90 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:33.364634 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 469/954] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jul 28 03:47:33 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:47:33.554865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4261de-484d-4abc-9a4e-87d1c32a7405 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:33.555202 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4261de-484d-4abc-9a4e-87d1c32a7405 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:33.555439 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4261de-484d-4abc-9a4e-87d1c32a7405 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:33.556966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 486/955] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:33.562129 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d978e02-956f-46be-b8d7-71378992cc52 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:33.562426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d978e02-956f-46be-b8d7-71378992cc52 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:33.562708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d978e02-956f-46be-b8d7-71378992cc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:33.582886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2d978e02-956f-46be-b8d7-71378992cc52 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:33.584105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2d978e02-956f-46be-b8d7-71378992cc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:33.653555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 470/956] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:33.680603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4261de-484d-4abc-9a4e-87d1c32a7405 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:33.788061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4261de-484d-4abc-9a4e-87d1c32a7405 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:33.799413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:33.825306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6fbfb-36ac-4e13-ad1e-0c8e324e4def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:33.826086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6fbfb-36ac-4e13-ad1e-0c8e324e4def None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:33.826749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6fbfb-36ac-4e13-ad1e-0c8e324e4def None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:33.827063 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6fbfb-36ac-4e13-ad1e-0c8e324e4def None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:33.827777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6fbfb-36ac-4e13-ad1e-0c8e324e4def None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:33.829517 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6fbfb-36ac-4e13-ad1e-0c8e324e4def None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:33.846927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 487/957] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:47:33 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:35.130520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fa167-ae87-4e08-a9fb-bb72928bd3c6 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.131576 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fa167-ae87-4e08-a9fb-bb72928bd3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.132058 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fa167-ae87-4e08-a9fb-bb72928bd3c6 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.133815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 471/958] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:35.139348 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556ae05-0d5e-4597-847d-3ddce9427e09 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.140168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556ae05-0d5e-4597-847d-3ddce9427e09 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.140653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556ae05-0d5e-4597-847d-3ddce9427e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.160140 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7556ae05-0d5e-4597-847d-3ddce9427e09 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:35.162217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7556ae05-0d5e-4597-847d-3ddce9427e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:35.242085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 488/959] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:35.266446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fa167-ae87-4e08-a9fb-bb72928bd3c6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:35.353010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:35.375570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fa167-ae87-4e08-a9fb-bb72928bd3c6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:35.391089 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d0926-7225-4fab-b8ae-68e00ec16db7 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.391377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d0926-7225-4fab-b8ae-68e00ec16db7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.391579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d0926-7225-4fab-b8ae-68e00ec16db7 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.393238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 489/960] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:35.398595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93146ca-0e7c-44f8-b20c-6ef688d0b595 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.398913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93146ca-0e7c-44f8-b20c-6ef688d0b595 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.399199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93146ca-0e7c-44f8-b20c-6ef688d0b595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.418831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a93146ca-0e7c-44f8-b20c-6ef688d0b595 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:35.419039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b801d-79a1-4211-92a7-de734dedf128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:35.419534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b801d-79a1-4211-92a7-de734dedf128 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.419821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b801d-79a1-4211-92a7-de734dedf128 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.420031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a93146ca-0e7c-44f8-b20c-6ef688d0b595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:35.420151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b801d-79a1-4211-92a7-de734dedf128 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.420772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b801d-79a1-4211-92a7-de734dedf128 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:35.422216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b801d-79a1-4211-92a7-de734dedf128 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:35.425488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: WARNING keystone.server.flask.common [None req-3e1b801d-79a1-4211-92a7-de734dedf128 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 03:47:35.439174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.sql.core [None req-3e1b801d-79a1-4211-92a7-de734dedf128 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 03:47:35.439174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 03:47:35.439174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [parameters: {'user_id': 'c47650ffec5d4936a26b934832199678', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 03:47:35.439174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80472) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3e1b801d-79a1-4211-92a7-de734dedf128 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 28 03:47:35.449214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 03:47:35.450179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application yield resource Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise self.value Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application session.commit() Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 03:47:35.451183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.flush() Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application compat.raise_( Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 03:47:35.452067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 03:47:35.452952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 03:47:35.453791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c47650ffec5d4936a26b934832199678', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:35.454611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 03:47:35.457429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 03:47:35.459700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 03:47:35.459700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 03:47:35.459700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 03:47:35.459700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 03:47:35.459700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 03:47:35.459700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:35.459700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 472/961] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Jul 28 03:47:35 2022] POST /identity/v3/users => generated 160 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:35.469450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b801d-79a1-4211-92a7-de734dedf128 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:35.504775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 490/962] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:35.527561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93146ca-0e7c-44f8-b20c-6ef688d0b595 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:35.556141 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:35.607642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b801d-79a1-4211-92a7-de734dedf128 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:35.637053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93146ca-0e7c-44f8-b20c-6ef688d0b595 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:35.645765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e35e26-a689-43f6-a64f-25ae834a000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:35.646354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e35e26-a689-43f6-a64f-25ae834a000d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.646618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e35e26-a689-43f6-a64f-25ae834a000d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.646918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e35e26-a689-43f6-a64f-25ae834a000d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.654601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e35e26-a689-43f6-a64f-25ae834a000d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:35.656372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e35e26-a689-43f6-a64f-25ae834a000d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-10e35e26-a689-43f6-a64f-25ae834a000d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:35.668851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:35.669808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:35.670838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:35.670838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:35.670838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:35.670838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:35.670838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:35.670838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:35.670838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:35.670838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:35.670838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 03:47:35.670838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:35.670838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 473/963] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:47:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:35.672073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813068c2-59d6-46be-82b2-e597c05cb422 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.672073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813068c2-59d6-46be-82b2-e597c05cb422 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.672073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813068c2-59d6-46be-82b2-e597c05cb422 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.675849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 474/964] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:35.676565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813068c2-59d6-46be-82b2-e597c05cb422 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:35.679484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:35.686145 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d88251-a06f-4540-897b-3684aaaa8b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:35.687530 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d88251-a06f-4540-897b-3684aaaa8b3d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.688041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d88251-a06f-4540-897b-3684aaaa8b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.688650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d88251-a06f-4540-897b-3684aaaa8b3d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.701038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d88251-a06f-4540-897b-3684aaaa8b3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-06d88251-a06f-4540-897b-3684aaaa8b3d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:35.707086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:35.713312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 491/965] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jul 28 03:47:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:35.714239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52cc1d-0e2a-4d57-aa4f-3f2e5184e518 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.715036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52cc1d-0e2a-4d57-aa4f-3f2e5184e518 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.715036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52cc1d-0e2a-4d57-aa4f-3f2e5184e518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.729905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4c52cc1d-0e2a-4d57-aa4f-3f2e5184e518 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:35.731130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4c52cc1d-0e2a-4d57-aa4f-3f2e5184e518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:35.792087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813068c2-59d6-46be-82b2-e597c05cb422 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:35.807770 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 492/966] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:35.807770 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52cc1d-0e2a-4d57-aa4f-3f2e5184e518 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:35.827217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:35.841701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c68f2-2e69-4401-a613-68cff69be7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:35.842322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c68f2-2e69-4401-a613-68cff69be7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.842604 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c68f2-2e69-4401-a613-68cff69be7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.842909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c68f2-2e69-4401-a613-68cff69be7e4 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.843690 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c68f2-2e69-4401-a613-68cff69be7e4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:35.843996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c68f2-2e69-4401-a613-68cff69be7e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:35.845429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c68f2-2e69-4401-a613-68cff69be7e4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:35.885053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 475/967] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jul 28 03:47:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:35.885053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c68f2-2e69-4401-a613-68cff69be7e4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:35.920170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52cc1d-0e2a-4d57-aa4f-3f2e5184e518 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:35.958148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:35.958610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac894e-7afd-4a83-8203-8c00f9f4f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:35.959286 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac894e-7afd-4a83-8203-8c00f9f4f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.959575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac894e-7afd-4a83-8203-8c00f9f4f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.959851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac894e-7afd-4a83-8203-8c00f9f4f4c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.960616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac894e-7afd-4a83-8203-8c00f9f4f4c4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:35.960898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac894e-7afd-4a83-8203-8c00f9f4f4c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:35.962313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac894e-7afd-4a83-8203-8c00f9f4f4c4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:35.972829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 493/968] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jul 28 03:47:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:35.974157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b6a2d-bb33-434c-bf4a-bfe8dbf777b8 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:35.974438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b6a2d-bb33-434c-bf4a-bfe8dbf777b8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:35.974715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b6a2d-bb33-434c-bf4a-bfe8dbf777b8 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:35.976205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 494/969] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:35.980944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b6a2d-bb33-434c-bf4a-bfe8dbf777b8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.013590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c68f2-2e69-4401-a613-68cff69be7e4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.052785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6380c3-378e-46e8-af18-e2e5b7270702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.053441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6380c3-378e-46e8-af18-e2e5b7270702 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.054631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6380c3-378e-46e8-af18-e2e5b7270702 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.054631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6380c3-378e-46e8-af18-e2e5b7270702 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.055250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6380c3-378e-46e8-af18-e2e5b7270702 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.056764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6380c3-378e-46e8-af18-e2e5b7270702 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ce6380c3-378e-46e8-af18-e2e5b7270702 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:36.068498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:36.069545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:36.070452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 476/970] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jul 28 03:47:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:36.075162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659eb14-e0f0-422b-a2ff-c90540f71474 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.075769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659eb14-e0f0-422b-a2ff-c90540f71474 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.076212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659eb14-e0f0-422b-a2ff-c90540f71474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.079299 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:36.100206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e659eb14-e0f0-422b-a2ff-c90540f71474 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:36.101753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e659eb14-e0f0-422b-a2ff-c90540f71474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:36.111372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b6a2d-bb33-434c-bf4a-bfe8dbf777b8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.149163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803c43a-1edb-4f43-9b27-3853ae54b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.149811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803c43a-1edb-4f43-9b27-3853ae54b702 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.150075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803c43a-1edb-4f43-9b27-3853ae54b702 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.150352 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803c43a-1edb-4f43-9b27-3853ae54b702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.160873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803c43a-1edb-4f43-9b27-3853ae54b702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.162588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803c43a-1edb-4f43-9b27-3853ae54b702 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e803c43a-1edb-4f43-9b27-3853ae54b702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:36.177545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:36.178614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:36.179556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:36.179556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:36.179556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:36.179556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:36.179556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 495/971] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:36.179847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 477/972] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:36.180755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803c43a-1edb-4f43-9b27-3853ae54b702 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.184470 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659eb14-e0f0-422b-a2ff-c90540f71474 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.243863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:36.282231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803c43a-1edb-4f43-9b27-3853ae54b702 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.289223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659eb14-e0f0-422b-a2ff-c90540f71474 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.310735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:36.318170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621efff-9531-44e6-8b46-a75bc64abd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.318776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621efff-9531-44e6-8b46-a75bc64abd1a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.319085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621efff-9531-44e6-8b46-a75bc64abd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.319356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621efff-9531-44e6-8b46-a75bc64abd1a None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.320026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621efff-9531-44e6-8b46-a75bc64abd1a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:36.320327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621efff-9531-44e6-8b46-a75bc64abd1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.321685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621efff-9531-44e6-8b46-a75bc64abd1a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.323532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf08006-5ed7-4f18-9da7-85219fdccd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.324148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf08006-5ed7-4f18-9da7-85219fdccd7b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.324433 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf08006-5ed7-4f18-9da7-85219fdccd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.324716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf08006-5ed7-4f18-9da7-85219fdccd7b None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.325401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf08006-5ed7-4f18-9da7-85219fdccd7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:36.325705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf08006-5ed7-4f18-9da7-85219fdccd7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.326460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 496/973] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jul 28 03:47:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:36.327093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf08006-5ed7-4f18-9da7-85219fdccd7b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.327265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621efff-9531-44e6-8b46-a75bc64abd1a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.342918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 478/974] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:36.342918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf08006-5ed7-4f18-9da7-85219fdccd7b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.431643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621efff-9531-44e6-8b46-a75bc64abd1a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.436649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf08006-5ed7-4f18-9da7-85219fdccd7b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.450422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:36.464738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26427fdc-d828-4f8c-984f-19b35152f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.465370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26427fdc-d828-4f8c-984f-19b35152f177 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.465651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26427fdc-d828-4f8c-984f-19b35152f177 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.465916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26427fdc-d828-4f8c-984f-19b35152f177 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.466530 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26427fdc-d828-4f8c-984f-19b35152f177 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.467919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26427fdc-d828-4f8c-984f-19b35152f177 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.472691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143488ba-ded9-428a-9ff8-f0b0d612fc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.473297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143488ba-ded9-428a-9ff8-f0b0d612fc44 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.473577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143488ba-ded9-428a-9ff8-f0b0d612fc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.473852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143488ba-ded9-428a-9ff8-f0b0d612fc44 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.474527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143488ba-ded9-428a-9ff8-f0b0d612fc44 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:36.474864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143488ba-ded9-428a-9ff8-f0b0d612fc44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.476200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143488ba-ded9-428a-9ff8-f0b0d612fc44 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-26427fdc-d828-4f8c-984f-19b35152f177 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:36.477118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:36.478100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:36.479001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 497/975] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:36.479818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26427fdc-d828-4f8c-984f-19b35152f177 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.490467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 479/976] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jul 28 03:47:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:36.490467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143488ba-ded9-428a-9ff8-f0b0d612fc44 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.543692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:36.586918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26427fdc-d828-4f8c-984f-19b35152f177 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.590128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143488ba-ded9-428a-9ff8-f0b0d612fc44 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.620589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290706ac-0ea9-4013-af6f-cb5b3bd3f4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.621219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290706ac-0ea9-4013-af6f-cb5b3bd3f4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.621502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290706ac-0ea9-4013-af6f-cb5b3bd3f4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.621775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290706ac-0ea9-4013-af6f-cb5b3bd3f4b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.623879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98042e2a-29ab-41d3-9a65-46578d4a7097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.624497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98042e2a-29ab-41d3-9a65-46578d4a7097 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.624782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98042e2a-29ab-41d3-9a65-46578d4a7097 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.625061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98042e2a-29ab-41d3-9a65-46578d4a7097 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.625742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98042e2a-29ab-41d3-9a65-46578d4a7097 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:36.626050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98042e2a-29ab-41d3-9a65-46578d4a7097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.628058 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98042e2a-29ab-41d3-9a65-46578d4a7097 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.629930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290706ac-0ea9-4013-af6f-cb5b3bd3f4b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.631388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290706ac-0ea9-4013-af6f-cb5b3bd3f4b3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.632782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 480/977] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:36.633545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98042e2a-29ab-41d3-9a65-46578d4a7097 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-290706ac-0ea9-4013-af6f-cb5b3bd3f4b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:36.647769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:36.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:36.649755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:36.649755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:36.649755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:36.649755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:36.649755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:36.649755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:36.649755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:36.649755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:36.649755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 03:47:36.649755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:36.649755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 498/978] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jul 28 03:47:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:36.650352 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290706ac-0ea9-4013-af6f-cb5b3bd3f4b3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.681420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:36.740662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98042e2a-29ab-41d3-9a65-46578d4a7097 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.752102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290706ac-0ea9-4013-af6f-cb5b3bd3f4b3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.769900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:36.773663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bd80b-252f-4e5d-8101-a011768e1528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.774418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bd80b-252f-4e5d-8101-a011768e1528 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.774703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bd80b-252f-4e5d-8101-a011768e1528 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.775007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bd80b-252f-4e5d-8101-a011768e1528 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.775711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79bd80b-252f-4e5d-8101-a011768e1528 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.777084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79bd80b-252f-4e5d-8101-a011768e1528 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.787291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740009f9-a1df-4684-8fed-c8c938c87319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.787904 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740009f9-a1df-4684-8fed-c8c938c87319 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.788184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740009f9-a1df-4684-8fed-c8c938c87319 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.788453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740009f9-a1df-4684-8fed-c8c938c87319 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.789245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740009f9-a1df-4684-8fed-c8c938c87319 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:36.789550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740009f9-a1df-4684-8fed-c8c938c87319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.791137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740009f9-a1df-4684-8fed-c8c938c87319 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.801497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 499/979] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:36.801497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740009f9-a1df-4684-8fed-c8c938c87319 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.801738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 481/980] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:47:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:36.802700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bd80b-252f-4e5d-8101-a011768e1528 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.912198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:36.918750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740009f9-a1df-4684-8fed-c8c938c87319 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.926939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bd80b-252f-4e5d-8101-a011768e1528 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:36.957343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b526e3-a11e-422d-a41f-07eb312daec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.957979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b526e3-a11e-422d-a41f-07eb312daec4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.958261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b526e3-a11e-422d-a41f-07eb312daec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.958535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b526e3-a11e-422d-a41f-07eb312daec4 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.959373 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b526e3-a11e-422d-a41f-07eb312daec4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:36.959680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b526e3-a11e-422d-a41f-07eb312daec4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.961111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b526e3-a11e-422d-a41f-07eb312daec4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.969180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52351d7d-016d-45b9-9329-d1e6ef8f9d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:36.969809 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52351d7d-016d-45b9-9329-d1e6ef8f9d50 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:36.970094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52351d7d-016d-45b9-9329-d1e6ef8f9d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:36.970368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52351d7d-016d-45b9-9329-d1e6ef8f9d50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:36.971169 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52351d7d-016d-45b9-9329-d1e6ef8f9d50 None admin] RBAC: Adding query filter params (interface=public, service_id=1556f8f041674da5bf5f8205a76ab649, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:36.972019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52351d7d-016d-45b9-9329-d1e6ef8f9d50 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:36.973381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52351d7d-016d-45b9-9329-d1e6ef8f9d50 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:36.980731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 500/981] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jul 28 03:47:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:36.980884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 482/982] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jul 28 03:47:36 2022] GET /identity/v3/endpoints?service_id=1556f8f041674da5bf5f8205a76ab649&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:36.986117 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b526e3-a11e-422d-a41f-07eb312daec4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:36.988422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52351d7d-016d-45b9-9329-d1e6ef8f9d50 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:37.030053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:37.095544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b526e3-a11e-422d-a41f-07eb312daec4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:37.110314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52351d7d-016d-45b9-9329-d1e6ef8f9d50 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:37.132427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bc49d-2023-4e28-8621-ee5efd7c604e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:37.133219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bc49d-2023-4e28-8621-ee5efd7c604e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:37.133610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bc49d-2023-4e28-8621-ee5efd7c604e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:37.133992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bc49d-2023-4e28-8621-ee5efd7c604e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:37.148415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cbe1d-fd26-4c55-bf02-547e16733ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:37.149030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cbe1d-fd26-4c55-bf02-547e16733ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:37.149310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cbe1d-fd26-4c55-bf02-547e16733ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:37.149583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cbe1d-fd26-4c55-bf02-547e16733ca6 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:37.150233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cbe1d-fd26-4c55-bf02-547e16733ca6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:37.151861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cbe1d-fd26-4c55-bf02-547e16733ca6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:37.166815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 483/983] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jul 28 03:47:36 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 03:47:37.212653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bc49d-2023-4e28-8621-ee5efd7c604e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:37.214286 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bc49d-2023-4e28-8621-ee5efd7c604e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:37.261562 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:37.327926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 501/984] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:36 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 03:47:37.334011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cbe1d-fd26-4c55-bf02-547e16733ca6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:37.360242 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:37.437777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cbe1d-fd26-4c55-bf02-547e16733ca6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:37.472898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e97fc-fedc-4f22-8360-2b051eec5f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:37.473828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e97fc-fedc-4f22-8360-2b051eec5f24 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:37.474314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e97fc-fedc-4f22-8360-2b051eec5f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:37.474760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e97fc-fedc-4f22-8360-2b051eec5f24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:37.485987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e97fc-fedc-4f22-8360-2b051eec5f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:37.488146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e97fc-fedc-4f22-8360-2b051eec5f24 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-e56e97fc-fedc-4f22-8360-2b051eec5f24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:37.502706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:37.503682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:37.504580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:37.504580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:37.504580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:37.504580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:37.504580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 484/985] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:37.510883 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bc49d-2023-4e28-8621-ee5efd7c604e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:37.619115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bc49d-2023-4e28-8621-ee5efd7c604e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:37.655393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac82318-1b33-44ac-85bb-31e47fddad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:37.656005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac82318-1b33-44ac-85bb-31e47fddad3c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:37.656285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac82318-1b33-44ac-85bb-31e47fddad3c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:37.656558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac82318-1b33-44ac-85bb-31e47fddad3c None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:37.657237 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac82318-1b33-44ac-85bb-31e47fddad3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:37.657539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac82318-1b33-44ac-85bb-31e47fddad3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:37.659042 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac82318-1b33-44ac-85bb-31e47fddad3c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:37.667768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:37.676569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 502/986] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:37.677795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e97fc-fedc-4f22-8360-2b051eec5f24 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:37.756220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:37.783075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e97fc-fedc-4f22-8360-2b051eec5f24 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:37.818176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b2ac8-c0a5-49b1-a12e-7781fc917d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:37.818845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b2ac8-c0a5-49b1-a12e-7781fc917d91 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:37.819137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b2ac8-c0a5-49b1-a12e-7781fc917d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:37.819414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b2ac8-c0a5-49b1-a12e-7781fc917d91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:37.826261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b2ac8-c0a5-49b1-a12e-7781fc917d91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:37.827731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b2ac8-c0a5-49b1-a12e-7781fc917d91 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5e3b2ac8-c0a5-49b1-a12e-7781fc917d91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:37.839967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:37.840822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:37.847011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:37.847011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:37.847011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:37.847011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:37.847011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:37.847011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:37.847011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:37.847011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:37.847011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 485/987] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:37.848272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac82318-1b33-44ac-85bb-31e47fddad3c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:37.952318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac82318-1b33-44ac-85bb-31e47fddad3c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:37.986370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fed3cb-8d4f-48e1-a6dc-69e096f4d969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:37.987590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fed3cb-8d4f-48e1-a6dc-69e096f4d969 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:37.988021 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fed3cb-8d4f-48e1-a6dc-69e096f4d969 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:37.988304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fed3cb-8d4f-48e1-a6dc-69e096f4d969 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:37.989056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fed3cb-8d4f-48e1-a6dc-69e096f4d969 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:37.989367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fed3cb-8d4f-48e1-a6dc-69e096f4d969 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:37.990820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fed3cb-8d4f-48e1-a6dc-69e096f4d969 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:38.003774 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 503/988] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:38.009532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b2ac8-c0a5-49b1-a12e-7781fc917d91 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:38.109925 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b2ac8-c0a5-49b1-a12e-7781fc917d91 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:38.143534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bdc89-bdac-4e06-95bd-a05ca52d3ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:38.144145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bdc89-bdac-4e06-95bd-a05ca52d3ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:38.144416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bdc89-bdac-4e06-95bd-a05ca52d3ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:38.144689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bdc89-bdac-4e06-95bd-a05ca52d3ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:38.145470 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bdc89-bdac-4e06-95bd-a05ca52d3ed2 None admin] RBAC: Adding query filter params (role.id=daec222fcf874e5fa9f48fc8192ea9a2, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=3b23317779d94f99bf804a172bb9c4ed) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:38.145782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bdc89-bdac-4e06-95bd-a05ca52d3ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:38.147227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bdc89-bdac-4e06-95bd-a05ca52d3ed2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:38.149041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:38.160213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 486/989] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:38 2022] GET /identity/v3/role_assignments?user.id=3b23317779d94f99bf804a172bb9c4ed&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=daec222fcf874e5fa9f48fc8192ea9a2 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:38.821605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc28ac-54e9-45e0-9f76-a29cf180e7d9 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:38.822152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc28ac-54e9-45e0-9f76-a29cf180e7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:38.822152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc28ac-54e9-45e0-9f76-a29cf180e7d9 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:38.823655 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 504/990] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:38.828680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e171c4-8a3e-41bc-9ca1-7efe036a1156 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:38.829056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e171c4-8a3e-41bc-9ca1-7efe036a1156 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:38.829235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e171c4-8a3e-41bc-9ca1-7efe036a1156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:38.848413 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-55e171c4-8a3e-41bc-9ca1-7efe036a1156 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:38.849590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-55e171c4-8a3e-41bc-9ca1-7efe036a1156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:38.917061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 487/991] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:38.938338 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc28ac-54e9-45e0-9f76-a29cf180e7d9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:38.998697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:39.048656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc28ac-54e9-45e0-9f76-a29cf180e7d9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:39.083096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553fe66-ad86-432c-8799-576298d9e002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:39.083658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553fe66-ad86-432c-8799-576298d9e002 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:39.083937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553fe66-ad86-432c-8799-576298d9e002 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:39.084209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553fe66-ad86-432c-8799-576298d9e002 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:39.084828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553fe66-ad86-432c-8799-576298d9e002 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:39.086177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553fe66-ad86-432c-8799-576298d9e002 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-0553fe66-ad86-432c-8799-576298d9e002 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:39.094925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:39.095914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:39.097323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:39.098861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 505/992] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jul 28 03:47:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:39.103042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e171c4-8a3e-41bc-9ca1-7efe036a1156 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:39.194883 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aec10a-7877-444b-9f7d-03eccd9d0b62 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:39.195174 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aec10a-7877-444b-9f7d-03eccd9d0b62 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:39.195448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aec10a-7877-444b-9f7d-03eccd9d0b62 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:39.196797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 506/993] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:47:39.201451 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717b2b6-3a5c-42c2-8abc-8e6232cde27d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:39.201741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717b2b6-3a5c-42c2-8abc-8e6232cde27d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:39.202014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717b2b6-3a5c-42c2-8abc-8e6232cde27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:39.204789 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e171c4-8a3e-41bc-9ca1-7efe036a1156 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:39.219941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e717b2b6-3a5c-42c2-8abc-8e6232cde27d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:39.221781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e717b2b6-3a5c-42c2-8abc-8e6232cde27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:39.234284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:39.243927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed01d84-8582-44c6-be53-48c18db0ce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:39.244683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01d84-8582-44c6-be53-48c18db0ce89 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:39.244965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01d84-8582-44c6-be53-48c18db0ce89 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:39.245235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01d84-8582-44c6-be53-48c18db0ce89 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:39.245904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed01d84-8582-44c6-be53-48c18db0ce89 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:39.246210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed01d84-8582-44c6-be53-48c18db0ce89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:39.247706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed01d84-8582-44c6-be53-48c18db0ce89 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:39.258120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 488/994] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jul 28 03:47:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:39.259587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed01d84-8582-44c6-be53-48c18db0ce89 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:39.294546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 507/995] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:39.318617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717b2b6-3a5c-42c2-8abc-8e6232cde27d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:39.372084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed01d84-8582-44c6-be53-48c18db0ce89 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:39.385151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:39.407930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63cd90-d652-4288-987f-344f68bd699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:39.408565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63cd90-d652-4288-987f-344f68bd699e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:39.408872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63cd90-d652-4288-987f-344f68bd699e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:39.409139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63cd90-d652-4288-987f-344f68bd699e None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:39.409985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63cd90-d652-4288-987f-344f68bd699e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:39.410288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63cd90-d652-4288-987f-344f68bd699e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:39.411942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63cd90-d652-4288-987f-344f68bd699e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:39.416953 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 489/996] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jul 28 03:47:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:39.422657 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63cd90-d652-4288-987f-344f68bd699e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:39.438623 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717b2b6-3a5c-42c2-8abc-8e6232cde27d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:39.476139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85985e58-ac0e-435d-9cd2-6e3aa24f698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:39.476778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985e58-ac0e-435d-9cd2-6e3aa24f698f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:39.477059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985e58-ac0e-435d-9cd2-6e3aa24f698f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:39.477333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985e58-ac0e-435d-9cd2-6e3aa24f698f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:39.477958 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85985e58-ac0e-435d-9cd2-6e3aa24f698f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:39.479780 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85985e58-ac0e-435d-9cd2-6e3aa24f698f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:39.482619 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-85985e58-ac0e-435d-9cd2-6e3aa24f698f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:39.490027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:39.491164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:39.492016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 508/997] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:39.497055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85985e58-ac0e-435d-9cd2-6e3aa24f698f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:39.532606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63cd90-d652-4288-987f-344f68bd699e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:39.568941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cc44f-d6e3-4667-9eba-05ad775b646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:39.569560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cc44f-d6e3-4667-9eba-05ad775b646d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:39.569842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cc44f-d6e3-4667-9eba-05ad775b646d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:39.570112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cc44f-d6e3-4667-9eba-05ad775b646d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:39.570903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cc44f-d6e3-4667-9eba-05ad775b646d None admin] RBAC: Adding query filter params (interface=admin, service_id=2cea1d4c39ef42cc87461a20e7cd3086, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:39.571214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cc44f-d6e3-4667-9eba-05ad775b646d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:39.572569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cc44f-d6e3-4667-9eba-05ad775b646d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:39.577746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 490/998] 10.222.0.46 () {56 vars in 1249 bytes} [Thu Jul 28 03:47:39 2022] GET /identity/v3/endpoints?service_id=2cea1d4c39ef42cc87461a20e7cd3086&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:39.583599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cc44f-d6e3-4667-9eba-05ad775b646d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:39.619638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85985e58-ac0e-435d-9cd2-6e3aa24f698f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:39.641808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:39.661991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dec19d-7805-4102-8650-b27ea1bd9921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:39.662620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dec19d-7805-4102-8650-b27ea1bd9921 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:39.662946 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dec19d-7805-4102-8650-b27ea1bd9921 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:39.663211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dec19d-7805-4102-8650-b27ea1bd9921 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:39.663892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dec19d-7805-4102-8650-b27ea1bd9921 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:39.664193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dec19d-7805-4102-8650-b27ea1bd9921 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:39.665554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dec19d-7805-4102-8650-b27ea1bd9921 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:39.670486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 509/999] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:39.676784 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dec19d-7805-4102-8650-b27ea1bd9921 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:39.693775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cc44f-d6e3-4667-9eba-05ad775b646d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:39.715931 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:39.733748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67118ba3-5be7-4aca-a1bf-da40dc3bbdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:39.734387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67118ba3-5be7-4aca-a1bf-da40dc3bbdee None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:39.734668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67118ba3-5be7-4aca-a1bf-da40dc3bbdee None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:39.734970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67118ba3-5be7-4aca-a1bf-da40dc3bbdee None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:39.735624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67118ba3-5be7-4aca-a1bf-da40dc3bbdee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:39.737203 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67118ba3-5be7-4aca-a1bf-da40dc3bbdee None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:39.743990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 491/1000] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jul 28 03:47:39 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 03:47:39.795486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dec19d-7805-4102-8650-b27ea1bd9921 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:39.833270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fcbe2-caa7-47f6-8636-842185ca2485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:39.833985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fcbe2-caa7-47f6-8636-842185ca2485 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:39.834243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fcbe2-caa7-47f6-8636-842185ca2485 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:39.834435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fcbe2-caa7-47f6-8636-842185ca2485 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:39.835167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fcbe2-caa7-47f6-8636-842185ca2485 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:39.835473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fcbe2-caa7-47f6-8636-842185ca2485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:39.836855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fcbe2-caa7-47f6-8636-842185ca2485 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:39.842772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 510/1001] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jul 28 03:47:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:39.848443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67118ba3-5be7-4aca-a1bf-da40dc3bbdee None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:39.959334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67118ba3-5be7-4aca-a1bf-da40dc3bbdee None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:39.993356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef79f9-5122-4e73-9c5a-dcf9e87d8aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:39.993967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef79f9-5122-4e73-9c5a-dcf9e87d8aa2 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:39.994246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef79f9-5122-4e73-9c5a-dcf9e87d8aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:39.994517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef79f9-5122-4e73-9c5a-dcf9e87d8aa2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:39.995266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef79f9-5122-4e73-9c5a-dcf9e87d8aa2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:39.996628 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef79f9-5122-4e73-9c5a-dcf9e87d8aa2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:40.022816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 492/1002] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:47:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:40.242726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9433f15-abe0-425b-b0f2-4a1bf4ef5565 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:40.243067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9433f15-abe0-425b-b0f2-4a1bf4ef5565 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:40.243269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9433f15-abe0-425b-b0f2-4a1bf4ef5565 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:40.244629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 511/1003] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:40.250001 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbef6a-6794-4e23-9180-620a0fdf565b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:40.250294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbef6a-6794-4e23-9180-620a0fdf565b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:40.250570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbef6a-6794-4e23-9180-620a0fdf565b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:40.270534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-99bbef6a-6794-4e23-9180-620a0fdf565b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:40.271779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-99bbef6a-6794-4e23-9180-620a0fdf565b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:40.309214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:40.351277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 493/1004] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:40 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:40.373436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9433f15-abe0-425b-b0f2-4a1bf4ef5565 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:40.426058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:40.483954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9433f15-abe0-425b-b0f2-4a1bf4ef5565 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:40.519746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bb2fe-596c-4615-90de-dae4322ae571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:40.520314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bb2fe-596c-4615-90de-dae4322ae571 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:40.520598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bb2fe-596c-4615-90de-dae4322ae571 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:40.520872 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bb2fe-596c-4615-90de-dae4322ae571 None admin] PATH_INFO: `/v3/services/image` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:40.521495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bb2fe-596c-4615-90de-dae4322ae571 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:40.522895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bb2fe-596c-4615-90de-dae4322ae571 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3a7bb2fe-596c-4615-90de-dae4322ae571 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:40.534295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:40.535430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 03:47:40.536565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:40.539018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 512/1005] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Jul 28 03:47:40 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:40.541218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbef6a-6794-4e23-9180-620a0fdf565b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:40.648975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbef6a-6794-4e23-9180-620a0fdf565b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:40.682964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ae6e0-38dd-4fb3-9063-55017ea1496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:40.683586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ae6e0-38dd-4fb3-9063-55017ea1496f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:40.683869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ae6e0-38dd-4fb3-9063-55017ea1496f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:40.684143 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ae6e0-38dd-4fb3-9063-55017ea1496f None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:40.684821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ae6e0-38dd-4fb3-9063-55017ea1496f None admin] RBAC: Adding query filter params (name=image) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:40.685127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ae6e0-38dd-4fb3-9063-55017ea1496f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:40.686493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ae6e0-38dd-4fb3-9063-55017ea1496f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:40.691769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 494/1006] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:40.697460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bb2fe-596c-4615-90de-dae4322ae571 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:40.800445 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bb2fe-596c-4615-90de-dae4322ae571 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:40.811300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:40.839533 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e745bf-4218-4a0e-9101-95afe0bd5b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:40.840188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e745bf-4218-4a0e-9101-95afe0bd5b33 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:40.840437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e745bf-4218-4a0e-9101-95afe0bd5b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:40.840704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e745bf-4218-4a0e-9101-95afe0bd5b33 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:40.841451 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e745bf-4218-4a0e-9101-95afe0bd5b33 None admin] RBAC: Adding query filter params (type=image) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:40.841759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e745bf-4218-4a0e-9101-95afe0bd5b33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:40.843229 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e745bf-4218-4a0e-9101-95afe0bd5b33 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:40.848199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 513/1007] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:40 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:41.779774 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7375d-cb5c-49db-bc1c-756ec5937a90 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:41.780311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7375d-cb5c-49db-bc1c-756ec5937a90 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:41.780311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7375d-cb5c-49db-bc1c-756ec5937a90 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:41.781691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 495/1008] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:41.787173 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998f534-26aa-491d-906e-6d93742781bd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:41.787406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998f534-26aa-491d-906e-6d93742781bd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:41.787679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998f534-26aa-491d-906e-6d93742781bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:41.806083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f998f534-26aa-491d-906e-6d93742781bd None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:41.807281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f998f534-26aa-491d-906e-6d93742781bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:41.877979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 514/1009] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:41 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:41.900369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7375d-cb5c-49db-bc1c-756ec5937a90 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:41.931498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:42.007119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7375d-cb5c-49db-bc1c-756ec5937a90 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:42.042665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53740e59-c34a-4d4e-8e3d-7512d5d75307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:42.043252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53740e59-c34a-4d4e-8e3d-7512d5d75307 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.043532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53740e59-c34a-4d4e-8e3d-7512d5d75307 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.043804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53740e59-c34a-4d4e-8e3d-7512d5d75307 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.044422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53740e59-c34a-4d4e-8e3d-7512d5d75307 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:42.045772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53740e59-c34a-4d4e-8e3d-7512d5d75307 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-53740e59-c34a-4d4e-8e3d-7512d5d75307 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:42.055267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:42.056351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 03:47:42.057225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:42.059100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 496/1010] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jul 28 03:47:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:42.062351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998f534-26aa-491d-906e-6d93742781bd None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:42.149637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b830a3-5008-464f-84fa-4bf35bc0c6de None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.149962 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b830a3-5008-464f-84fa-4bf35bc0c6de None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.150193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b830a3-5008-464f-84fa-4bf35bc0c6de None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.151722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 497/1011] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:47:42.156468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563e211-8417-44aa-a8f1-b90cba2a99e9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.156755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563e211-8417-44aa-a8f1-b90cba2a99e9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.157032 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563e211-8417-44aa-a8f1-b90cba2a99e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.170151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998f534-26aa-491d-906e-6d93742781bd None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:42.172647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:42.176636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d563e211-8417-44aa-a8f1-b90cba2a99e9 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:42.177807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d563e211-8417-44aa-a8f1-b90cba2a99e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:42.209680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd25d9-bcc8-471e-908a-ab64c6f58a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:42.210390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd25d9-bcc8-471e-908a-ab64c6f58a28 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.210767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd25d9-bcc8-471e-908a-ab64c6f58a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.211163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd25d9-bcc8-471e-908a-ab64c6f58a28 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.211846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd25d9-bcc8-471e-908a-ab64c6f58a28 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:42.212153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd25d9-bcc8-471e-908a-ab64c6f58a28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:42.213788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd25d9-bcc8-471e-908a-ab64c6f58a28 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:42.219357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 515/1012] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jul 28 03:47:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:42.224568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd25d9-bcc8-471e-908a-ab64c6f58a28 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:42.249264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 498/1013] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:42 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:42.272569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563e211-8417-44aa-a8f1-b90cba2a99e9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:42.296709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:42.333216 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd25d9-bcc8-471e-908a-ab64c6f58a28 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:42.369785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabd789-7f1f-4520-981f-f87aceca835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:42.370404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabd789-7f1f-4520-981f-f87aceca835a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.370687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabd789-7f1f-4520-981f-f87aceca835a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.370988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabd789-7f1f-4520-981f-f87aceca835a None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.371667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabd789-7f1f-4520-981f-f87aceca835a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:42.371972 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabd789-7f1f-4520-981f-f87aceca835a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:42.372193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563e211-8417-44aa-a8f1-b90cba2a99e9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:42.373328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabd789-7f1f-4520-981f-f87aceca835a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:42.379660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 516/1014] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jul 28 03:47:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:42.385740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabd789-7f1f-4520-981f-f87aceca835a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:42.408289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb246a-316c-4465-b887-9371808f0adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:42.409508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb246a-316c-4465-b887-9371808f0adb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.409788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb246a-316c-4465-b887-9371808f0adb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.410062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb246a-316c-4465-b887-9371808f0adb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.419792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:42.421684 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb246a-316c-4465-b887-9371808f0adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-edcb246a-316c-4465-b887-9371808f0adb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:42.427399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:42.428334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:42.429226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 499/1015] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jul 28 03:47:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:42.434344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb246a-316c-4465-b887-9371808f0adb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:42.501163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabd789-7f1f-4520-981f-f87aceca835a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:42.522750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:42.539631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb246a-316c-4465-b887-9371808f0adb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:42.543713 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a8d4d-fdcf-486c-8bfa-0acbeab7b981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:42.544339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a8d4d-fdcf-486c-8bfa-0acbeab7b981 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.544618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a8d4d-fdcf-486c-8bfa-0acbeab7b981 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.544888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a8d4d-fdcf-486c-8bfa-0acbeab7b981 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.545596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a8d4d-fdcf-486c-8bfa-0acbeab7b981 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:42.547041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a8d4d-fdcf-486c-8bfa-0acbeab7b981 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:42.566971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 517/1016] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:47:42 2022] POST /identity/v3/endpoints => generated 325 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:42.599807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820aaf2-d621-4aff-a1b1-4b315ee92dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:42.600546 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820aaf2-d621-4aff-a1b1-4b315ee92dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.600891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820aaf2-d621-4aff-a1b1-4b315ee92dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.601234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820aaf2-d621-4aff-a1b1-4b315ee92dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.602035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820aaf2-d621-4aff-a1b1-4b315ee92dd9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:42.602434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820aaf2-d621-4aff-a1b1-4b315ee92dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:42.603956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820aaf2-d621-4aff-a1b1-4b315ee92dd9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:42.619461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 500/1017] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jul 28 03:47:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:42.620887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a8d4d-fdcf-486c-8bfa-0acbeab7b981 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:42.728585 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:42.741391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a8d4d-fdcf-486c-8bfa-0acbeab7b981 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:42.778436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0dc85-b4b2-494b-8500-525fa7f77466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:42.779109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0dc85-b4b2-494b-8500-525fa7f77466 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.779393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0dc85-b4b2-494b-8500-525fa7f77466 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.779668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0dc85-b4b2-494b-8500-525fa7f77466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.791315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0dc85-b4b2-494b-8500-525fa7f77466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:42.792760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0dc85-b4b2-494b-8500-525fa7f77466 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-05c0dc85-b4b2-494b-8500-525fa7f77466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:42.808248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:42.809147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:42.810056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:42.810056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:42.810056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:42.810056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:42.810056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 518/1018] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:42.816250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820aaf2-d621-4aff-a1b1-4b315ee92dd9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:42.897580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc881cb-b46f-475b-9e03-ecbb3d83508d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.898106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc881cb-b46f-475b-9e03-ecbb3d83508d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.898197 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc881cb-b46f-475b-9e03-ecbb3d83508d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.899664 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 519/1019] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:42.904386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928eddf-aa8d-4d27-a08c-24565d342c9f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.904681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928eddf-aa8d-4d27-a08c-24565d342c9f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.904968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928eddf-aa8d-4d27-a08c-24565d342c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.921647 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820aaf2-d621-4aff-a1b1-4b315ee92dd9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:42.924999 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c928eddf-aa8d-4d27-a08c-24565d342c9f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:42.926312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c928eddf-aa8d-4d27-a08c-24565d342c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:42.959421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededccdf-c33e-4906-a79f-6bfc541d74f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:42.960320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededccdf-c33e-4906-a79f-6bfc541d74f3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:42.960796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededccdf-c33e-4906-a79f-6bfc541d74f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:42.961352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededccdf-c33e-4906-a79f-6bfc541d74f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:42.962349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededccdf-c33e-4906-a79f-6bfc541d74f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:42.962652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededccdf-c33e-4906-a79f-6bfc541d74f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:42.964041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededccdf-c33e-4906-a79f-6bfc541d74f3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:42.981637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 501/1020] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:42.982699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededccdf-c33e-4906-a79f-6bfc541d74f3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:42.985458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:43.002561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 520/1021] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:42 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:43.024968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928eddf-aa8d-4d27-a08c-24565d342c9f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:43.093296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededccdf-c33e-4906-a79f-6bfc541d74f3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:43.103371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:43.130486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928eddf-aa8d-4d27-a08c-24565d342c9f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:43.131437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047a6f9-c2c6-4889-ba81-f1e9d1c8086f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:43.132079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047a6f9-c2c6-4889-ba81-f1e9d1c8086f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:43.132362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047a6f9-c2c6-4889-ba81-f1e9d1c8086f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:43.132633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047a6f9-c2c6-4889-ba81-f1e9d1c8086f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:43.139913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047a6f9-c2c6-4889-ba81-f1e9d1c8086f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:43.141337 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047a6f9-c2c6-4889-ba81-f1e9d1c8086f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a047a6f9-c2c6-4889-ba81-f1e9d1c8086f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:43.153951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:43.155118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:43.156080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:43.156080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:43.156080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:43.156080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:43.156080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:43.156080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:43.156080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:43.156080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:43.156080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:43.156080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:43.156080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 502/1022] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jul 28 03:47:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:43.160845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047a6f9-c2c6-4889-ba81-f1e9d1c8086f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:43.171191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60ffcb-ea9c-4a0a-b2b4-1f2194d526ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:43.172031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60ffcb-ea9c-4a0a-b2b4-1f2194d526ab None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:43.172316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60ffcb-ea9c-4a0a-b2b4-1f2194d526ab None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:43.172590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60ffcb-ea9c-4a0a-b2b4-1f2194d526ab None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:43.173268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60ffcb-ea9c-4a0a-b2b4-1f2194d526ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:43.175336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60ffcb-ea9c-4a0a-b2b4-1f2194d526ab None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:43.194838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 521/1023] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:47:43 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:47:43.234280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:43.281825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047a6f9-c2c6-4889-ba81-f1e9d1c8086f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:43.325177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009644d-71d2-4db7-82b6-1f24b1754d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:43.325806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009644d-71d2-4db7-82b6-1f24b1754d34 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:43.326083 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009644d-71d2-4db7-82b6-1f24b1754d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:43.326355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009644d-71d2-4db7-82b6-1f24b1754d34 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:43.327188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009644d-71d2-4db7-82b6-1f24b1754d34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:43.327551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009644d-71d2-4db7-82b6-1f24b1754d34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:43.329111 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009644d-71d2-4db7-82b6-1f24b1754d34 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:43.350697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 503/1024] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jul 28 03:47:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:43.355812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60ffcb-ea9c-4a0a-b2b4-1f2194d526ab None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:43.462809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60ffcb-ea9c-4a0a-b2b4-1f2194d526ab None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:43.496714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f105bc8-8a6d-4107-a6f8-c8a670229453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:43.497326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f105bc8-8a6d-4107-a6f8-c8a670229453 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:43.497608 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f105bc8-8a6d-4107-a6f8-c8a670229453 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:43.497880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f105bc8-8a6d-4107-a6f8-c8a670229453 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:43.500466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:43.574824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f105bc8-8a6d-4107-a6f8-c8a670229453 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:43.576268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f105bc8-8a6d-4107-a6f8-c8a670229453 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:43.679409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 522/1025] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 03:47:43.680998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009644d-71d2-4db7-82b6-1f24b1754d34 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:43.782453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009644d-71d2-4db7-82b6-1f24b1754d34 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:43.799644 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:43.818980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a22bf0-041e-46db-97e5-e83ba5474bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:43.819600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a22bf0-041e-46db-97e5-e83ba5474bad None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:43.819882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a22bf0-041e-46db-97e5-e83ba5474bad None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:43.820149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a22bf0-041e-46db-97e5-e83ba5474bad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:43.829959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a22bf0-041e-46db-97e5-e83ba5474bad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:43.831570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a22bf0-041e-46db-97e5-e83ba5474bad None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-83a22bf0-041e-46db-97e5-e83ba5474bad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:43.845825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:43.851903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:43.852885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:43.852885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:43.852885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:43.852885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:43.852885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 504/1026] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:43.853176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f105bc8-8a6d-4107-a6f8-c8a670229453 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:43.887357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:43.957566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f105bc8-8a6d-4107-a6f8-c8a670229453 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:43.991893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1a214-95ea-4188-90da-f17b4dade2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:43.992506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1a214-95ea-4188-90da-f17b4dade2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:43.992786 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1a214-95ea-4188-90da-f17b4dade2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:43.993059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1a214-95ea-4188-90da-f17b4dade2a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:43.993744 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1a214-95ea-4188-90da-f17b4dade2a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:43.994046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1a214-95ea-4188-90da-f17b4dade2a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:43.995430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1a214-95ea-4188-90da-f17b4dade2a8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:44.007012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 523/1027] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:44.012470 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a22bf0-041e-46db-97e5-e83ba5474bad None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:44.113924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a22bf0-041e-46db-97e5-e83ba5474bad None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:44.146724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d27e8-a6e0-4fe0-96ac-544d72486fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:44.147379 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d27e8-a6e0-4fe0-96ac-544d72486fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:44.147658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d27e8-a6e0-4fe0-96ac-544d72486fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:44.147928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d27e8-a6e0-4fe0-96ac-544d72486fe6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:44.154544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d27e8-a6e0-4fe0-96ac-544d72486fe6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:44.156000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d27e8-a6e0-4fe0-96ac-544d72486fe6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a41d27e8-a6e0-4fe0-96ac-544d72486fe6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:44.167668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:44.168601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:44.169566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:44.169566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:44.169566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:44.169566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:44.169566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:44.169566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:44.169566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:44.169566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:44.171109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 505/1028] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:44.175474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1a214-95ea-4188-90da-f17b4dade2a8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:44.270190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:44.278054 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1a214-95ea-4188-90da-f17b4dade2a8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:44.311577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae42d5d-1165-4f17-b9e5-f63a2a643425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:44.312191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae42d5d-1165-4f17-b9e5-f63a2a643425 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:44.312470 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae42d5d-1165-4f17-b9e5-f63a2a643425 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:44.312742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae42d5d-1165-4f17-b9e5-f63a2a643425 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:44.313485 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae42d5d-1165-4f17-b9e5-f63a2a643425 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:44.313789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae42d5d-1165-4f17-b9e5-f63a2a643425 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:44.315773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae42d5d-1165-4f17-b9e5-f63a2a643425 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:44.331810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 524/1029] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:44.334141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d27e8-a6e0-4fe0-96ac-544d72486fe6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:44.367015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:44.442649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d27e8-a6e0-4fe0-96ac-544d72486fe6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:44.477171 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8388-5cd9-4403-ba5e-7ebf0a7bfdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:44.477786 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8388-5cd9-4403-ba5e-7ebf0a7bfdab None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:44.478069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8388-5cd9-4403-ba5e-7ebf0a7bfdab None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:44.478342 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8388-5cd9-4403-ba5e-7ebf0a7bfdab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:44.479164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8388-5cd9-4403-ba5e-7ebf0a7bfdab None admin] RBAC: Adding query filter params (role.id=daec222fcf874e5fa9f48fc8192ea9a2, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=162ba16d643747c287bfe8a8e59df235) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:44.479478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8388-5cd9-4403-ba5e-7ebf0a7bfdab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:44.480909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8388-5cd9-4403-ba5e-7ebf0a7bfdab None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:44.495169 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 506/1030] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:44 2022] GET /identity/v3/role_assignments?user.id=162ba16d643747c287bfe8a8e59df235&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=daec222fcf874e5fa9f48fc8192ea9a2 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:44.648597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467ba95-da06-4cb1-9bff-425878f7a26a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:44.648921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467ba95-da06-4cb1-9bff-425878f7a26a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:44.649153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467ba95-da06-4cb1-9bff-425878f7a26a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:44.650677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 525/1031] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:44.655621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58cd6-ba81-4200-8b42-ceff047c2a2f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:44.655918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58cd6-ba81-4200-8b42-ceff047c2a2f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:44.656198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58cd6-ba81-4200-8b42-ceff047c2a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:44.675440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-31d58cd6-ba81-4200-8b42-ceff047c2a2f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:44.676636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-31d58cd6-ba81-4200-8b42-ceff047c2a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:44.744147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 507/1032] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:44.767178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467ba95-da06-4cb1-9bff-425878f7a26a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:44.872029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467ba95-da06-4cb1-9bff-425878f7a26a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:44.906479 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf7877-9abc-4c95-ad52-9e54e88a2255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:44.907200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf7877-9abc-4c95-ad52-9e54e88a2255 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:44.907485 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf7877-9abc-4c95-ad52-9e54e88a2255 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:44.907760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf7877-9abc-4c95-ad52-9e54e88a2255 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:44.909184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf7877-9abc-4c95-ad52-9e54e88a2255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:44.910615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf7877-9abc-4c95-ad52-9e54e88a2255 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:44.917274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 526/1033] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:44.919286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d58cd6-ba81-4200-8b42-ceff047c2a2f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:44.943673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:45.022517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d58cd6-ba81-4200-8b42-ceff047c2a2f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:45.056853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734dc88-5560-492b-99bd-8b6e4547dc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:45.057464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734dc88-5560-492b-99bd-8b6e4547dc48 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:45.057742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734dc88-5560-492b-99bd-8b6e4547dc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:45.058013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734dc88-5560-492b-99bd-8b6e4547dc48 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:45.058683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734dc88-5560-492b-99bd-8b6e4547dc48 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:45.060142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734dc88-5560-492b-99bd-8b6e4547dc48 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:45.094811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 508/1034] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:47:44 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:45.270649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb5b93-1535-429a-92a8-473dabb4ccb8 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:45.271179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb5b93-1535-429a-92a8-473dabb4ccb8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:45.271179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb5b93-1535-429a-92a8-473dabb4ccb8 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:45.272524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 527/1035] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:45.277779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9a40-e1a6-424e-937a-853a771ab070 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:45.278078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9a40-e1a6-424e-937a-853a771ab070 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:45.278356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9a40-e1a6-424e-937a-853a771ab070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:45.297539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-52fb9a40-e1a6-424e-937a-853a771ab070 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:45.298738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-52fb9a40-e1a6-424e-937a-853a771ab070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:45.367190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 509/1036] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:45.390387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb5b93-1535-429a-92a8-473dabb4ccb8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:45.416323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:45.497476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb5b93-1535-429a-92a8-473dabb4ccb8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:45.533307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b49ba-5133-469c-b4b6-e5110763a994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:45.534037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b49ba-5133-469c-b4b6-e5110763a994 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:45.534315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b49ba-5133-469c-b4b6-e5110763a994 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:45.534586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b49ba-5133-469c-b4b6-e5110763a994 None admin] PATH_INFO: `/v3/services/image` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:45.536096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b49ba-5133-469c-b4b6-e5110763a994 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:45.537461 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b49ba-5133-469c-b4b6-e5110763a994 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3d0b49ba-5133-469c-b4b6-e5110763a994 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:45.546246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:45.547497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:45.550826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 528/1037] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Jul 28 03:47:45 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:45.554926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb9a40-e1a6-424e-937a-853a771ab070 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:45.557845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:45.663014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb9a40-e1a6-424e-937a-853a771ab070 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:45.697492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbb206-17a8-4397-a807-07dfc973ceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:45.698109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbb206-17a8-4397-a807-07dfc973ceb4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:45.698395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbb206-17a8-4397-a807-07dfc973ceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:45.698671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbb206-17a8-4397-a807-07dfc973ceb4 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:45.699451 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbb206-17a8-4397-a807-07dfc973ceb4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:45.699762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbb206-17a8-4397-a807-07dfc973ceb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:45.701126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbb206-17a8-4397-a807-07dfc973ceb4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:45.705985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 510/1038] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:45.711874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b49ba-5133-469c-b4b6-e5110763a994 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:45.797815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:45.813758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b49ba-5133-469c-b4b6-e5110763a994 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:45.848476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f6733-2474-430c-8370-ba6558fc91a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:45.849090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f6733-2474-430c-8370-ba6558fc91a1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:45.849370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f6733-2474-430c-8370-ba6558fc91a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:45.849640 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f6733-2474-430c-8370-ba6558fc91a1 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:45.850312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f6733-2474-430c-8370-ba6558fc91a1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:45.850614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f6733-2474-430c-8370-ba6558fc91a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:45.851990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f6733-2474-430c-8370-ba6558fc91a1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:45.861669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 529/1039] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:45.863323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbb206-17a8-4397-a807-07dfc973ceb4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:45.934187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:45.969901 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbb206-17a8-4397-a807-07dfc973ceb4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:46.005185 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504506d-6151-48c8-bea1-3baef79be80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:46.005802 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504506d-6151-48c8-bea1-3baef79be80a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:46.006086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504506d-6151-48c8-bea1-3baef79be80a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:46.006360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504506d-6151-48c8-bea1-3baef79be80a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:46.007148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504506d-6151-48c8-bea1-3baef79be80a None admin] RBAC: Adding query filter params (interface=public, service_id=18f44b89078447e397b3655f26c68ea2, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:46.007523 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504506d-6151-48c8-bea1-3baef79be80a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:46.008883 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504506d-6151-48c8-bea1-3baef79be80a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:46.019500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 511/1040] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jul 28 03:47:45 2022] GET /identity/v3/endpoints?service_id=18f44b89078447e397b3655f26c68ea2&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:46.020741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f6733-2474-430c-8370-ba6558fc91a1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:46.126156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f6733-2474-430c-8370-ba6558fc91a1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:46.162294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659887b-f1d3-44f4-8562-d5f045d1ff10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:46.162943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659887b-f1d3-44f4-8562-d5f045d1ff10 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:46.163253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659887b-f1d3-44f4-8562-d5f045d1ff10 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:46.163532 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659887b-f1d3-44f4-8562-d5f045d1ff10 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:46.164195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659887b-f1d3-44f4-8562-d5f045d1ff10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:46.165562 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659887b-f1d3-44f4-8562-d5f045d1ff10 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:46.172483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 530/1041] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jul 28 03:47:46 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:47:46.472841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c70549-a5ab-4612-89c7-3790984f4ea1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:46.473187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c70549-a5ab-4612-89c7-3790984f4ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:46.473310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c70549-a5ab-4612-89c7-3790984f4ea1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:46.474819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 512/1042] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:46.479710 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7c63a-3583-42de-b2a5-2f1b4bc03263 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:46.480144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7c63a-3583-42de-b2a5-2f1b4bc03263 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:46.480305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7c63a-3583-42de-b2a5-2f1b4bc03263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:46.498040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-98b7c63a-3583-42de-b2a5-2f1b4bc03263 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:46.499221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-98b7c63a-3583-42de-b2a5-2f1b4bc03263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:46.509617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:46.574828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 531/1043] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:46.595560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c70549-a5ab-4612-89c7-3790984f4ea1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:46.700396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c70549-a5ab-4612-89c7-3790984f4ea1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:46.730181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:46.736443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746f25d-cc8c-4cbc-9301-447694237d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:46.737649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746f25d-cc8c-4cbc-9301-447694237d1b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:46.737922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746f25d-cc8c-4cbc-9301-447694237d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:46.738193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746f25d-cc8c-4cbc-9301-447694237d1b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:46.748320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746f25d-cc8c-4cbc-9301-447694237d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f746f25d-cc8c-4cbc-9301-447694237d1b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:46.753218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:46.754156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:46.755041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 513/1044] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jul 28 03:47:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:46.761654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7c63a-3583-42de-b2a5-2f1b4bc03263 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:46.862824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7c63a-3583-42de-b2a5-2f1b4bc03263 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:46.896982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e45e82-8708-46ce-8d9a-76f89437c148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:46.897598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e45e82-8708-46ce-8d9a-76f89437c148 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:46.897878 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e45e82-8708-46ce-8d9a-76f89437c148 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:46.898152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e45e82-8708-46ce-8d9a-76f89437c148 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:46.898891 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e45e82-8708-46ce-8d9a-76f89437c148 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:46.899196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e45e82-8708-46ce-8d9a-76f89437c148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:46.900560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e45e82-8708-46ce-8d9a-76f89437c148 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:46.914263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 532/1045] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jul 28 03:47:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:46.915585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746f25d-cc8c-4cbc-9301-447694237d1b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:47.018155 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746f25d-cc8c-4cbc-9301-447694237d1b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:47.053495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea761c70-3821-4277-af9e-72b8fbfb8340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:47.054135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea761c70-3821-4277-af9e-72b8fbfb8340 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:47.054414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea761c70-3821-4277-af9e-72b8fbfb8340 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:47.054688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea761c70-3821-4277-af9e-72b8fbfb8340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:47.065075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea761c70-3821-4277-af9e-72b8fbfb8340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:47.066508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea761c70-3821-4277-af9e-72b8fbfb8340 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ea761c70-3821-4277-af9e-72b8fbfb8340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:47.081608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:47.082596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:47.083513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:47.083513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:47.083513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:47.083513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:47.084286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 514/1046] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:47.088600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e45e82-8708-46ce-8d9a-76f89437c148 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:47.099560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:47.149736 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f67b59-8187-4f0a-9b22-9fc0d0b76f84 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:47.150079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f67b59-8187-4f0a-9b22-9fc0d0b76f84 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:47.150305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f67b59-8187-4f0a-9b22-9fc0d0b76f84 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:47.151691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 515/1047] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:47.157279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f77375-ab9c-4cd6-8e14-da1aae412bcb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:47.157565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f77375-ab9c-4cd6-8e14-da1aae412bcb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:47.157840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f77375-ab9c-4cd6-8e14-da1aae412bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:47.178773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-68f77375-ab9c-4cd6-8e14-da1aae412bcb None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:47.179996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-68f77375-ab9c-4cd6-8e14-da1aae412bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:47.195954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e45e82-8708-46ce-8d9a-76f89437c148 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:47.223214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:47.232758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cf45d-28c8-4556-816a-5e4e6bcecd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:47.233380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cf45d-28c8-4556-816a-5e4e6bcecd4e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:47.233659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cf45d-28c8-4556-816a-5e4e6bcecd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:47.233977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cf45d-28c8-4556-816a-5e4e6bcecd4e None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:47.234677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cf45d-28c8-4556-816a-5e4e6bcecd4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:47.235008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cf45d-28c8-4556-816a-5e4e6bcecd4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:47.236620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cf45d-28c8-4556-816a-5e4e6bcecd4e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:47.253779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 533/1048] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:47.253950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 516/1049] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:47.254953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cf45d-28c8-4556-816a-5e4e6bcecd4e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:47.277023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f77375-ab9c-4cd6-8e14-da1aae412bcb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:47.328375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:47.367204 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cf45d-28c8-4556-816a-5e4e6bcecd4e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:47.386261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f77375-ab9c-4cd6-8e14-da1aae412bcb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:47.404282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252773c-cf9e-49ea-a72b-e3b2e7531cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:47.404928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252773c-cf9e-49ea-a72b-e3b2e7531cac None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:47.405213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252773c-cf9e-49ea-a72b-e3b2e7531cac None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:47.405488 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252773c-cf9e-49ea-a72b-e3b2e7531cac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:47.414351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252773c-cf9e-49ea-a72b-e3b2e7531cac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:47.415837 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252773c-cf9e-49ea-a72b-e3b2e7531cac None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:47.421973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff114843-6b26-4d13-bf77-406166d4f6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:47.422615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff114843-6b26-4d13-bf77-406166d4f6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:47.422972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff114843-6b26-4d13-bf77-406166d4f6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:47.423256 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff114843-6b26-4d13-bf77-406166d4f6c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:47.424441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff114843-6b26-4d13-bf77-406166d4f6c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:47.425860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff114843-6b26-4d13-bf77-406166d4f6c3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:47.427653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 517/1050] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e252773c-cf9e-49ea-a72b-e3b2e7531cac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:47.431279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:47.432170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:47.433107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:47.433107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:47.433107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:47.433107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:47.433107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:47.433107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:47.433107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:47.433107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:47.433107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:47.433107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:47.433107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 534/1051] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jul 28 03:47:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:47.433688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff114843-6b26-4d13-bf77-406166d4f6c3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:47.439188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252773c-cf9e-49ea-a72b-e3b2e7531cac None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:47.483236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:47.545190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff114843-6b26-4d13-bf77-406166d4f6c3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:47.545940 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252773c-cf9e-49ea-a72b-e3b2e7531cac None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:47.580600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:47.582646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773b7ee-e9cf-4db3-b1dc-c04fc0a09996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:47.583227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b7ee-e9cf-4db3-b1dc-c04fc0a09996 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:47.583511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b7ee-e9cf-4db3-b1dc-c04fc0a09996 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:47.583868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b7ee-e9cf-4db3-b1dc-c04fc0a09996 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:47.584681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773b7ee-e9cf-4db3-b1dc-c04fc0a09996 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:47.586137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773b7ee-e9cf-4db3-b1dc-c04fc0a09996 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:47.590334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5300291-9c06-4821-83e6-b50cd69d310e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:47.591032 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5300291-9c06-4821-83e6-b50cd69d310e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:47.591315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5300291-9c06-4821-83e6-b50cd69d310e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:47.591635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5300291-9c06-4821-83e6-b50cd69d310e None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:47.592430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5300291-9c06-4821-83e6-b50cd69d310e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:47.592736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5300291-9c06-4821-83e6-b50cd69d310e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:47.594146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5300291-9c06-4821-83e6-b50cd69d310e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:47.614885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 518/1052] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:47:47 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:47.619592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 535/1053] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jul 28 03:47:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:47.625465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773b7ee-e9cf-4db3-b1dc-c04fc0a09996 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:47.728172 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:47.731843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773b7ee-e9cf-4db3-b1dc-c04fc0a09996 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:47.770043 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef811c-8ecc-4d88-bcb7-064ac610a9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:47.770711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef811c-8ecc-4d88-bcb7-064ac610a9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:47.771024 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef811c-8ecc-4d88-bcb7-064ac610a9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:47.771301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef811c-8ecc-4d88-bcb7-064ac610a9a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:47.782965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef811c-8ecc-4d88-bcb7-064ac610a9a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:47.785189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef811c-8ecc-4d88-bcb7-064ac610a9a4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8cef811c-8ecc-4d88-bcb7-064ac610a9a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:47.801346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:47.802458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:47.803597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:47.803597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:47.803597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:47.803597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:47.803597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 519/1054] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:47.809519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5300291-9c06-4821-83e6-b50cd69d310e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:47.919069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5300291-9c06-4821-83e6-b50cd69d310e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:47.957092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f3ea6-90a4-4eae-962a-bb8f5229b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:47.958055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f3ea6-90a4-4eae-962a-bb8f5229b3de None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:47.958456 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f3ea6-90a4-4eae-962a-bb8f5229b3de None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:47.958911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f3ea6-90a4-4eae-962a-bb8f5229b3de None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:47.960007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f3ea6-90a4-4eae-962a-bb8f5229b3de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:47.960554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f3ea6-90a4-4eae-962a-bb8f5229b3de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:47.963091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f3ea6-90a4-4eae-962a-bb8f5229b3de None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:47.983936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 536/1055] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:47.985210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef811c-8ecc-4d88-bcb7-064ac610a9a4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:48.085130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef811c-8ecc-4d88-bcb7-064ac610a9a4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:48.119038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535da3e9-52f2-460a-a26e-0cac086e39f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:48.119689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535da3e9-52f2-460a-a26e-0cac086e39f1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:48.119975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535da3e9-52f2-460a-a26e-0cac086e39f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:48.120254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535da3e9-52f2-460a-a26e-0cac086e39f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:48.127496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535da3e9-52f2-460a-a26e-0cac086e39f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:48.128964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535da3e9-52f2-460a-a26e-0cac086e39f1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-535da3e9-52f2-460a-a26e-0cac086e39f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:48.140237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:48.141116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:48.142017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:48.142017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:48.142017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:48.142017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:48.142017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:48.142017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:48.142017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:48.142017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:48.142017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 520/1056] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:48.150827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f3ea6-90a4-4eae-962a-bb8f5229b3de None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:48.201061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:48.202286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dabda-570f-4a86-80a5-cb2069420675 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:48.202481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dabda-570f-4a86-80a5-cb2069420675 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:48.202751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dabda-570f-4a86-80a5-cb2069420675 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:48.204368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 521/1057] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:47:48.209964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e703-a215-4291-bc79-c3b795ab3866 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:48.210250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e703-a215-4291-bc79-c3b795ab3866 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:48.210523 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e703-a215-4291-bc79-c3b795ab3866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:48.229912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e6d4e703-a215-4291-bc79-c3b795ab3866 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:48.231138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e6d4e703-a215-4291-bc79-c3b795ab3866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:48.234008 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:48.264986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f3ea6-90a4-4eae-962a-bb8f5229b3de None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:48.305302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96421966-e199-46dd-941b-b349ae416fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:48.305928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96421966-e199-46dd-941b-b349ae416fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:48.306215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96421966-e199-46dd-941b-b349ae416fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:48.306495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96421966-e199-46dd-941b-b349ae416fd9 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:48.307274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96421966-e199-46dd-941b-b349ae416fd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:48.307603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96421966-e199-46dd-941b-b349ae416fd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:48.309006 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96421966-e199-46dd-941b-b349ae416fd9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:48.310838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 522/1058] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:48.330828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 537/1059] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:48.332459 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96421966-e199-46dd-941b-b349ae416fd9 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:48.332591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4e703-a215-4291-bc79-c3b795ab3866 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:48.439691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96421966-e199-46dd-941b-b349ae416fd9 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:48.440130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4e703-a215-4291-bc79-c3b795ab3866 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:48.443608 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:48.476913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8dfab-fd00-4e39-91a0-a963dbe8c5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:48.477646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8dfab-fd00-4e39-91a0-a963dbe8c5bd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:48.478038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8dfab-fd00-4e39-91a0-a963dbe8c5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:48.478395 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8dfab-fd00-4e39-91a0-a963dbe8c5bd None admin] PATH_INFO: `/v3/services/image` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:48.479193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8dfab-fd00-4e39-91a0-a963dbe8c5bd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:48.480637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8dfab-fd00-4e39-91a0-a963dbe8c5bd None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:48.481596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6e560-15c8-415f-94c9-0706f1968a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:48.482519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6e560-15c8-415f-94c9-0706f1968a76 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:48.482929 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6e560-15c8-415f-94c9-0706f1968a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:48.483555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6e560-15c8-415f-94c9-0706f1968a76 None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/162ba16d643747c287bfe8a8e59df235/roles/daec222fcf874e5fa9f48fc8192ea9a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:48.486071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6e560-15c8-415f-94c9-0706f1968a76 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=162ba16d643747c287bfe8a8e59df235, role_id=daec222fcf874e5fa9f48fc8192ea9a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:48.488344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6e560-15c8-415f-94c9-0706f1968a76 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-6fc8dfab-fd00-4e39-91a0-a963dbe8c5bd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:48.489995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:48.491161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:48.492036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 538/1060] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Jul 28 03:47:48 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:48.497100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8dfab-fd00-4e39-91a0-a963dbe8c5bd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:48.504489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:48.511372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 523/1061] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:47:48 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/162ba16d643747c287bfe8a8e59df235/roles/daec222fcf874e5fa9f48fc8192ea9a2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:47:48.614753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8dfab-fd00-4e39-91a0-a963dbe8c5bd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:48.651046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee28994-6bd2-4515-8fda-3e8a3b3c828b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:48.651834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee28994-6bd2-4515-8fda-3e8a3b3c828b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:48.652235 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee28994-6bd2-4515-8fda-3e8a3b3c828b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:48.652605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee28994-6bd2-4515-8fda-3e8a3b3c828b None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:48.653395 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee28994-6bd2-4515-8fda-3e8a3b3c828b None admin] RBAC: Adding query filter params (name=image) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:48.653807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee28994-6bd2-4515-8fda-3e8a3b3c828b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:48.655306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee28994-6bd2-4515-8fda-3e8a3b3c828b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:48.662820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 539/1062] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:48.667704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6e560-15c8-415f-94c9-0706f1968a76 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:48.773133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6e560-15c8-415f-94c9-0706f1968a76 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:48.807997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ecd64-b3aa-4eaa-ad51-0f9ed492912e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:48.808615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ecd64-b3aa-4eaa-ad51-0f9ed492912e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:48.808894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ecd64-b3aa-4eaa-ad51-0f9ed492912e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:48.809169 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ecd64-b3aa-4eaa-ad51-0f9ed492912e None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:48.809839 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ecd64-b3aa-4eaa-ad51-0f9ed492912e None admin] RBAC: Adding query filter params (type=image) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:48.810138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ecd64-b3aa-4eaa-ad51-0f9ed492912e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:48.811520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ecd64-b3aa-4eaa-ad51-0f9ed492912e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:48.817291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 524/1063] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:48.823002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee28994-6bd2-4515-8fda-3e8a3b3c828b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:48.856088 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:48.928037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee28994-6bd2-4515-8fda-3e8a3b3c828b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:48.961918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bc9c8-b29d-4651-bbeb-691ea5a4bf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:48.962537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bc9c8-b29d-4651-bbeb-691ea5a4bf6e None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:48.962838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bc9c8-b29d-4651-bbeb-691ea5a4bf6e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:48.963113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bc9c8-b29d-4651-bbeb-691ea5a4bf6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:48.963824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bc9c8-b29d-4651-bbeb-691ea5a4bf6e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:48.965177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bc9c8-b29d-4651-bbeb-691ea5a4bf6e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:48.985006 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 540/1064] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:47:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:47:49.742560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37573e78-87ca-4273-b20f-08775c197071 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:49.743090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37573e78-87ca-4273-b20f-08775c197071 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:49.743167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37573e78-87ca-4273-b20f-08775c197071 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:49.744540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 525/1065] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:49.749799 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd520c2-2c70-4917-83d9-6ef461d0f244 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:49.750200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd520c2-2c70-4917-83d9-6ef461d0f244 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:49.750361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd520c2-2c70-4917-83d9-6ef461d0f244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:49.769904 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6fd520c2-2c70-4917-83d9-6ef461d0f244 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:49.771117 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6fd520c2-2c70-4917-83d9-6ef461d0f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:49.843590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 541/1066] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:49.867055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37573e78-87ca-4273-b20f-08775c197071 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:49.932344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:49.973653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37573e78-87ca-4273-b20f-08775c197071 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:50.008220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b65de-a6ce-44ae-8e6e-31221ea155ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:50.008857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b65de-a6ce-44ae-8e6e-31221ea155ec None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:50.009140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b65de-a6ce-44ae-8e6e-31221ea155ec None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:50.009413 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b65de-a6ce-44ae-8e6e-31221ea155ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:50.020337 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b65de-a6ce-44ae-8e6e-31221ea155ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-502b65de-a6ce-44ae-8e6e-31221ea155ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:50.025266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:50.026174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:50.027409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:50.030858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 526/1067] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:50.032716 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd520c2-2c70-4917-83d9-6ef461d0f244 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:50.048333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:50.141186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd520c2-2c70-4917-83d9-6ef461d0f244 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:50.176810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fa2d1-046e-494e-ae83-2f59c1cde604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:50.177697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa2d1-046e-494e-ae83-2f59c1cde604 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:50.178128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa2d1-046e-494e-ae83-2f59c1cde604 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:50.178559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa2d1-046e-494e-ae83-2f59c1cde604 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:50.179451 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fa2d1-046e-494e-ae83-2f59c1cde604 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:50.180074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fa2d1-046e-494e-ae83-2f59c1cde604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:50.181612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fa2d1-046e-494e-ae83-2f59c1cde604 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:50.190649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 542/1068] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:47:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:50.196516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b65de-a6ce-44ae-8e6e-31221ea155ec None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:50.300920 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b65de-a6ce-44ae-8e6e-31221ea155ec None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:50.324569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:50.337084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa482d-26dc-4830-92a8-e4a3c6fdad50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:50.337725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa482d-26dc-4830-92a8-e4a3c6fdad50 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:50.338007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa482d-26dc-4830-92a8-e4a3c6fdad50 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:50.338282 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa482d-26dc-4830-92a8-e4a3c6fdad50 None admin] PATH_INFO: `/v3/users/bade6f2784e547afac484a0d4fbda596` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:50.351474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa482d-26dc-4830-92a8-e4a3c6fdad50 None admin] RBAC: Authorizing `identity:get_user(user_id=bade6f2784e547afac484a0d4fbda596)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:50.352913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa482d-26dc-4830-92a8-e4a3c6fdad50 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:50.359550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 527/1069] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:47:50 2022] GET /identity/v3/users/bade6f2784e547afac484a0d4fbda596 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:50.360509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fa2d1-046e-494e-ae83-2f59c1cde604 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:50.439844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:50.464630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fa2d1-046e-494e-ae83-2f59c1cde604 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:50.498090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a489e2-3df6-4c24-90bf-7c5d7365e01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:50.498711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a489e2-3df6-4c24-90bf-7c5d7365e01d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:50.499020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a489e2-3df6-4c24-90bf-7c5d7365e01d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:50.499299 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a489e2-3df6-4c24-90bf-7c5d7365e01d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:50.527405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a055b92-241c-46ab-a157-c70b5bf31df5 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:50.527703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a055b92-241c-46ab-a157-c70b5bf31df5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:50.527961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a055b92-241c-46ab-a157-c70b5bf31df5 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:50.530948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 528/1070] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:47:50.534818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab5019-c45c-4c10-93f4-5711879ee757 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:50.536407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab5019-c45c-4c10-93f4-5711879ee757 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:50.536922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab5019-c45c-4c10-93f4-5711879ee757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:50.563109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-27ab5019-c45c-4c10-93f4-5711879ee757 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:50.564358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-27ab5019-c45c-4c10-93f4-5711879ee757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:50.576938 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a489e2-3df6-4c24-90bf-7c5d7365e01d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:50.578389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a489e2-3df6-4c24-90bf-7c5d7365e01d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:50.638812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 529/1071] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:50.656996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:50.662982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab5019-c45c-4c10-93f4-5711879ee757 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:50.689804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 543/1072] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 03:47:50.691775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a489e2-3df6-4c24-90bf-7c5d7365e01d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:50.763101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab5019-c45c-4c10-93f4-5711879ee757 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:50.776976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:50.801982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c21051-e2c6-49ad-a312-a0b416bfc714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:50.802663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c21051-e2c6-49ad-a312-a0b416bfc714 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:50.802984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c21051-e2c6-49ad-a312-a0b416bfc714 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:50.803436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c21051-e2c6-49ad-a312-a0b416bfc714 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:50.807613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a489e2-3df6-4c24-90bf-7c5d7365e01d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:50.814440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c21051-e2c6-49ad-a312-a0b416bfc714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b1c21051-e2c6-49ad-a312-a0b416bfc714 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:50.819365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:50.820266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:50.821084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 530/1073] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jul 28 03:47:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:50.826404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c21051-e2c6-49ad-a312-a0b416bfc714 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:50.847345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee39034-36d3-4e0c-b3eb-0a2a8c44098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:50.847980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee39034-36d3-4e0c-b3eb-0a2a8c44098e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:50.848264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee39034-36d3-4e0c-b3eb-0a2a8c44098e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:50.848540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee39034-36d3-4e0c-b3eb-0a2a8c44098e None admin] PATH_INFO: `/v3/projects/0f2ee9ffc86f418680d23d388b3543d9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:50.849683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee39034-36d3-4e0c-b3eb-0a2a8c44098e None admin] RBAC: Authorizing `identity:get_project(project_id=0f2ee9ffc86f418680d23d388b3543d9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:50.851137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee39034-36d3-4e0c-b3eb-0a2a8c44098e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:50.852795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 544/1074] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:47:50 2022] GET /identity/v3/projects/0f2ee9ffc86f418680d23d388b3543d9 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:50.859053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee39034-36d3-4e0c-b3eb-0a2a8c44098e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:50.907834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:50.937061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c21051-e2c6-49ad-a312-a0b416bfc714 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:50.964015 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee39034-36d3-4e0c-b3eb-0a2a8c44098e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:50.979626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990149d-db37-4506-b7c1-3e823715a2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:50.980535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990149d-db37-4506-b7c1-3e823715a2ed None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:50.980994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990149d-db37-4506-b7c1-3e823715a2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:50.981461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990149d-db37-4506-b7c1-3e823715a2ed None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:50.982308 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990149d-db37-4506-b7c1-3e823715a2ed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:50.982830 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990149d-db37-4506-b7c1-3e823715a2ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:50.984573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990149d-db37-4506-b7c1-3e823715a2ed None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:50.995354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 531/1075] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jul 28 03:47:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:51.000532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990149d-db37-4506-b7c1-3e823715a2ed None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:51.006948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51a5e7-c750-4d24-8ab0-1bfee927d975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:51.007574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a5e7-c750-4d24-8ab0-1bfee927d975 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:51.007857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a5e7-c750-4d24-8ab0-1bfee927d975 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:51.008134 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a5e7-c750-4d24-8ab0-1bfee927d975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:51.009002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51a5e7-c750-4d24-8ab0-1bfee927d975 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=0f2ee9ffc86f418680d23d388b3543d9, user.id=bade6f2784e547afac484a0d4fbda596) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:51.009316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51a5e7-c750-4d24-8ab0-1bfee927d975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:51.010747 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51a5e7-c750-4d24-8ab0-1bfee927d975 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:51.016709 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 545/1076] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:50 2022] GET /identity/v3/role_assignments?user.id=bade6f2784e547afac484a0d4fbda596&scope.project.id=0f2ee9ffc86f418680d23d388b3543d9&role.id=a37ae24c516b48c69e13721c013134a2 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:51.044933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828a768-4b1c-41b5-a225-29dfaf550a2d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:51.045241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828a768-4b1c-41b5-a225-29dfaf550a2d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:51.045523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828a768-4b1c-41b5-a225-29dfaf550a2d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:51.047156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 546/1077] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:47:51.053430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a237a-6efb-46d8-bd4e-263aa136fa40 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:51.053736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a237a-6efb-46d8-bd4e-263aa136fa40 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:51.053985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a237a-6efb-46d8-bd4e-263aa136fa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:51.054997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:51.077349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7e6a237a-6efb-46d8-bd4e-263aa136fa40 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:51.078611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7e6a237a-6efb-46d8-bd4e-263aa136fa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:51.133042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990149d-db37-4506-b7c1-3e823715a2ed None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:51.153718 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 547/1078] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:47:51.174180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f9d05-4940-4a5a-a548-2bbfd2f45909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:51.175026 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f9d05-4940-4a5a-a548-2bbfd2f45909 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:51.175318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f9d05-4940-4a5a-a548-2bbfd2f45909 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:51.175594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f9d05-4940-4a5a-a548-2bbfd2f45909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:51.175693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a237a-6efb-46d8-bd4e-263aa136fa40 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:51.187372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f9d05-4940-4a5a-a548-2bbfd2f45909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:51.188855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f9d05-4940-4a5a-a548-2bbfd2f45909 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-242f9d05-4940-4a5a-a548-2bbfd2f45909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:51.204270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:51.205781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:51.206659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:51.206659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:51.206659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:51.206659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:51.206925 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 532/1079] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:51.212057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f9d05-4940-4a5a-a548-2bbfd2f45909 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:51.226457 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:51.290220 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a237a-6efb-46d8-bd4e-263aa136fa40 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:51.305750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:51.323070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f9d05-4940-4a5a-a548-2bbfd2f45909 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:51.329849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb0510-b85c-4cdc-a56b-75eea36091e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:51.330624 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb0510-b85c-4cdc-a56b-75eea36091e1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:51.330953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb0510-b85c-4cdc-a56b-75eea36091e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:51.331227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb0510-b85c-4cdc-a56b-75eea36091e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:51.360317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af3c5a-65fc-459f-b95c-ae3d5c1f410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:51.360938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3c5a-65fc-459f-b95c-ae3d5c1f410c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:51.361838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3c5a-65fc-459f-b95c-ae3d5c1f410c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:51.362267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3c5a-65fc-459f-b95c-ae3d5c1f410c None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:51.363003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af3c5a-65fc-459f-b95c-ae3d5c1f410c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:51.363387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af3c5a-65fc-459f-b95c-ae3d5c1f410c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:51.364770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af3c5a-65fc-459f-b95c-ae3d5c1f410c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:51.378404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 533/1080] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:51.383487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af3c5a-65fc-459f-b95c-ae3d5c1f410c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:51.409657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb0510-b85c-4cdc-a56b-75eea36091e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:51.411609 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb0510-b85c-4cdc-a56b-75eea36091e1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:51.452373 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:51.487634 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af3c5a-65fc-459f-b95c-ae3d5c1f410c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:51.521684 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 548/1081] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:47:51.523048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45718dcb-4fc9-4d32-b7f8-f77be8e5effd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:51.523700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45718dcb-4fc9-4d32-b7f8-f77be8e5effd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:51.523983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45718dcb-4fc9-4d32-b7f8-f77be8e5effd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:51.524257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45718dcb-4fc9-4d32-b7f8-f77be8e5effd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:51.528120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb0510-b85c-4cdc-a56b-75eea36091e1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:51.534300 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45718dcb-4fc9-4d32-b7f8-f77be8e5effd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:51.534572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45718dcb-4fc9-4d32-b7f8-f77be8e5effd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-45718dcb-4fc9-4d32-b7f8-f77be8e5effd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:47:51.550932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:51.551910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:51.552967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:51.552967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:51.552967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:47:51.552967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:47:51.552967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:47:51.552967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:47:51.552967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:47:51.552967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:47:51.552967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 03:47:51.552967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:51.555625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 534/1082] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jul 28 03:47:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:51.558301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45718dcb-4fc9-4d32-b7f8-f77be8e5effd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:51.569675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:51.644226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb0510-b85c-4cdc-a56b-75eea36091e1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:51.667359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45718dcb-4fc9-4d32-b7f8-f77be8e5effd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:51.683110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32eadf2-5003-4ba2-91d2-39f545446a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:51.683826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32eadf2-5003-4ba2-91d2-39f545446a9d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:51.684107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32eadf2-5003-4ba2-91d2-39f545446a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:51.684382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32eadf2-5003-4ba2-91d2-39f545446a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:51.694274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:51.699892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32eadf2-5003-4ba2-91d2-39f545446a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:51.701691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32eadf2-5003-4ba2-91d2-39f545446a9d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:51.703727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f43e14-079a-42b3-a96a-f97ace3e6603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:51.704346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f43e14-079a-42b3-a96a-f97ace3e6603 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:51.704630 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f43e14-079a-42b3-a96a-f97ace3e6603 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:51.704969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f43e14-079a-42b3-a96a-f97ace3e6603 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:51.705770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f43e14-079a-42b3-a96a-f97ace3e6603 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:51.706076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f43e14-079a-42b3-a96a-f97ace3e6603 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:51.707511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f43e14-079a-42b3-a96a-f97ace3e6603 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-f32eadf2-5003-4ba2-91d2-39f545446a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:51.717462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:51.718495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:51.719549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:51.719549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:51.719549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:51.719549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:51.719549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 549/1083] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:51.724675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32eadf2-5003-4ba2-91d2-39f545446a9d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:51.731799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 535/1084] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jul 28 03:47:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:51.737346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f43e14-079a-42b3-a96a-f97ace3e6603 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:51.813111 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:51.831420 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32eadf2-5003-4ba2-91d2-39f545446a9d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:51.853942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f43e14-079a-42b3-a96a-f97ace3e6603 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:51.867455 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09037a-aa78-44b0-a63d-965d69c0edc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:51.868074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09037a-aa78-44b0-a63d-965d69c0edc2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:51.868358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09037a-aa78-44b0-a63d-965d69c0edc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:51.868631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09037a-aa78-44b0-a63d-965d69c0edc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:51.869320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09037a-aa78-44b0-a63d-965d69c0edc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:51.869624 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09037a-aa78-44b0-a63d-965d69c0edc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:51.871021 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09037a-aa78-44b0-a63d-965d69c0edc2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:51.883401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 550/1085] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:51.889542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e4b46-0e03-4f62-aa72-975388134844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:51.890189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e4b46-0e03-4f62-aa72-975388134844 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:51.890475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e4b46-0e03-4f62-aa72-975388134844 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:51.890751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e4b46-0e03-4f62-aa72-975388134844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:51.968141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e4b46-0e03-4f62-aa72-975388134844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:51.969612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e4b46-0e03-4f62-aa72-975388134844 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:52.037500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:52.078315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 536/1086] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 03:47:52.084330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09037a-aa78-44b0-a63d-965d69c0edc2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:52.143204 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:52.197390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09037a-aa78-44b0-a63d-965d69c0edc2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:52.232115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d65f3-97e4-421a-b3ab-fda6b91f2dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:52.232748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d65f3-97e4-421a-b3ab-fda6b91f2dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:52.233031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d65f3-97e4-421a-b3ab-fda6b91f2dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:52.233307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d65f3-97e4-421a-b3ab-fda6b91f2dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:52.243468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d65f3-97e4-421a-b3ab-fda6b91f2dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:52.244905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d65f3-97e4-421a-b3ab-fda6b91f2dd2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-925d65f3-97e4-421a-b3ab-fda6b91f2dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:47:52.260249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:52.261158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:52.262083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:47:52.262083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:47:52.262083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:47:52.262083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:52.262083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 551/1087] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:47:52.267044 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e4b46-0e03-4f62-aa72-975388134844 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:52.364492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e4b46-0e03-4f62-aa72-975388134844 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:52.397661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028da61-fea7-48c7-85cc-79d444f1a27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:52.398269 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028da61-fea7-48c7-85cc-79d444f1a27b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:52.398553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028da61-fea7-48c7-85cc-79d444f1a27b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:52.398863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028da61-fea7-48c7-85cc-79d444f1a27b None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:52.399552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028da61-fea7-48c7-85cc-79d444f1a27b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:52.399856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028da61-fea7-48c7-85cc-79d444f1a27b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:52.401213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028da61-fea7-48c7-85cc-79d444f1a27b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:52.413033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 537/1088] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:52.418643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d65f3-97e4-421a-b3ab-fda6b91f2dd2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:52.515302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d65f3-97e4-421a-b3ab-fda6b91f2dd2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:52.516806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:52.551628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85f0a6-6653-44af-bc06-8d82575f384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:52.552268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f0a6-6653-44af-bc06-8d82575f384c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:52.552549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f0a6-6653-44af-bc06-8d82575f384c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:52.552823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f0a6-6653-44af-bc06-8d82575f384c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:52.559643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85f0a6-6653-44af-bc06-8d82575f384c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:52.561067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85f0a6-6653-44af-bc06-8d82575f384c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-bf85f0a6-6653-44af-bc06-8d82575f384c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:47:52.572853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:52.573921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:52.579582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:47:52.579582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:47:52.579582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:47:52.579582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:47:52.579582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:47:52.579582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:47:52.579582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:47:52.579582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:52.579582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 552/1089] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:47:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:52.580727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028da61-fea7-48c7-85cc-79d444f1a27b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:52.605540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:52.679428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028da61-fea7-48c7-85cc-79d444f1a27b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:52.712813 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56164fa-1ffc-4222-8ffa-03cb3596b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:52.713434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56164fa-1ffc-4222-8ffa-03cb3596b7ed None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:52.713712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56164fa-1ffc-4222-8ffa-03cb3596b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:52.713984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56164fa-1ffc-4222-8ffa-03cb3596b7ed None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:52.714728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56164fa-1ffc-4222-8ffa-03cb3596b7ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:52.715061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56164fa-1ffc-4222-8ffa-03cb3596b7ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:52.716488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56164fa-1ffc-4222-8ffa-03cb3596b7ed None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:52.728807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 538/1090] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:47:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:52.734734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85f0a6-6653-44af-bc06-8d82575f384c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:52.834741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85f0a6-6653-44af-bc06-8d82575f384c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:52.867858 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029a569-5ffd-4fd7-bd80-967302c0ae75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:52.868470 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029a569-5ffd-4fd7-bd80-967302c0ae75 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:52.868757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029a569-5ffd-4fd7-bd80-967302c0ae75 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:52.869037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029a569-5ffd-4fd7-bd80-967302c0ae75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:52.869824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029a569-5ffd-4fd7-bd80-967302c0ae75 None admin] RBAC: Adding query filter params (role.id=daec222fcf874e5fa9f48fc8192ea9a2, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=162ba16d643747c287bfe8a8e59df235) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:52.870138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029a569-5ffd-4fd7-bd80-967302c0ae75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:52.871594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029a569-5ffd-4fd7-bd80-967302c0ae75 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:52.878199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 553/1091] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:52 2022] GET /identity/v3/role_assignments?user.id=162ba16d643747c287bfe8a8e59df235&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=daec222fcf874e5fa9f48fc8192ea9a2 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:53.021530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe38a22-e468-4364-8383-b0bf1b655471 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:53.022014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe38a22-e468-4364-8383-b0bf1b655471 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:53.022122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe38a22-e468-4364-8383-b0bf1b655471 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:53.023700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 539/1092] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:53.028966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0883f-8fce-48d9-bee7-eaa56c10a6e6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:53.029262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0883f-8fce-48d9-bee7-eaa56c10a6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:53.029544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0883f-8fce-48d9-bee7-eaa56c10a6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:53.049213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6bf0883f-8fce-48d9-bee7-eaa56c10a6e6 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:53.050487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6bf0883f-8fce-48d9-bee7-eaa56c10a6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:53.072657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:53.122074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 554/1093] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:53.144562 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe38a22-e468-4364-8383-b0bf1b655471 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:53.233873 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:53.253773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe38a22-e468-4364-8383-b0bf1b655471 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:53.289247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b39339-3c5f-402f-9540-1ebc2a72c688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:53.289820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b39339-3c5f-402f-9540-1ebc2a72c688 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:53.290104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b39339-3c5f-402f-9540-1ebc2a72c688 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:53.290378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b39339-3c5f-402f-9540-1ebc2a72c688 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:53.300711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b39339-3c5f-402f-9540-1ebc2a72c688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-39b39339-3c5f-402f-9540-1ebc2a72c688 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:53.305602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:53.306703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:47:53.307626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 540/1094] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:53.315005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0883f-8fce-48d9-bee7-eaa56c10a6e6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:53.420381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0883f-8fce-48d9-bee7-eaa56c10a6e6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:53.455778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7482ab-0c99-4f1b-9023-4b6f08ef17ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:53.456401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7482ab-0c99-4f1b-9023-4b6f08ef17ef None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:53.456689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7482ab-0c99-4f1b-9023-4b6f08ef17ef None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:53.456966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7482ab-0c99-4f1b-9023-4b6f08ef17ef None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:53.457690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7482ab-0c99-4f1b-9023-4b6f08ef17ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:53.457998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7482ab-0c99-4f1b-9023-4b6f08ef17ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:53.459421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7482ab-0c99-4f1b-9023-4b6f08ef17ef None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:53.467618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 555/1095] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:47:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:53.474447 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b39339-3c5f-402f-9540-1ebc2a72c688 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:53.582266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b39339-3c5f-402f-9540-1ebc2a72c688 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:53.616672 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db9544-6fd3-4a4b-b31d-4084b597cfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:53.617312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db9544-6fd3-4a4b-b31d-4084b597cfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:53.617590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db9544-6fd3-4a4b-b31d-4084b597cfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:53.617862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db9544-6fd3-4a4b-b31d-4084b597cfc4 None admin] PATH_INFO: `/v3/users/bade6f2784e547afac484a0d4fbda596` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:53.619121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db9544-6fd3-4a4b-b31d-4084b597cfc4 None admin] RBAC: Authorizing `identity:get_user(user_id=bade6f2784e547afac484a0d4fbda596)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:53.620541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db9544-6fd3-4a4b-b31d-4084b597cfc4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:53.622445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 541/1096] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:47:53 2022] GET /identity/v3/users/bade6f2784e547afac484a0d4fbda596 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:53.628307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7482ab-0c99-4f1b-9023-4b6f08ef17ef None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:53.648805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:53.733850 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7482ab-0c99-4f1b-9023-4b6f08ef17ef None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:53.767176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84cd32-6e8b-4d93-9de1-20fd711ef101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:53.767849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cd32-6e8b-4d93-9de1-20fd711ef101 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:53.768130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cd32-6e8b-4d93-9de1-20fd711ef101 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:53.768402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cd32-6e8b-4d93-9de1-20fd711ef101 None admin] PATH_INFO: `/v3/projects/0f2ee9ffc86f418680d23d388b3543d9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:53.769541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84cd32-6e8b-4d93-9de1-20fd711ef101 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2ee9ffc86f418680d23d388b3543d9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:53.771017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84cd32-6e8b-4d93-9de1-20fd711ef101 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:53.772665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 556/1097] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:47:53 2022] GET /identity/v3/projects/0f2ee9ffc86f418680d23d388b3543d9 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:53.778714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db9544-6fd3-4a4b-b31d-4084b597cfc4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:53.793880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:53.875307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db9544-6fd3-4a4b-b31d-4084b597cfc4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:53.907336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0faa9d1-1b93-4090-b6fd-2f4c37a1e494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:53.908502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0faa9d1-1b93-4090-b6fd-2f4c37a1e494 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:53.908782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0faa9d1-1b93-4090-b6fd-2f4c37a1e494 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:53.909050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0faa9d1-1b93-4090-b6fd-2f4c37a1e494 None admin] PATH_INFO: `/v3/projects/0f2ee9ffc86f418680d23d388b3543d9/users/bade6f2784e547afac484a0d4fbda596/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:53.910982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0faa9d1-1b93-4090-b6fd-2f4c37a1e494 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f2ee9ffc86f418680d23d388b3543d9, user_id=bade6f2784e547afac484a0d4fbda596, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:53.912657 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0faa9d1-1b93-4090-b6fd-2f4c37a1e494 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:53.924540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 542/1098] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:47:53 2022] PUT /identity/v3/projects/0f2ee9ffc86f418680d23d388b3543d9/users/bade6f2784e547afac484a0d4fbda596/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 03:47:55.883825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5235d5-c31a-4322-a204-f2bbebba3cd6 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:55.884350 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5235d5-c31a-4322-a204-f2bbebba3cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:55.884350 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5235d5-c31a-4322-a204-f2bbebba3cd6 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:55.885710 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 557/1099] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:55.890868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0f6e6-69a3-4314-b4a5-bfb161d3a42f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:55.891266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0f6e6-69a3-4314-b4a5-bfb161d3a42f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:55.891422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0f6e6-69a3-4314-b4a5-bfb161d3a42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:55.909064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cfe0f6e6-69a3-4314-b4a5-bfb161d3a42f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:55.910234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cfe0f6e6-69a3-4314-b4a5-bfb161d3a42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:55.975440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 543/1100] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:55.998344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5235d5-c31a-4322-a204-f2bbebba3cd6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:56.082302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:56.097593 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5235d5-c31a-4322-a204-f2bbebba3cd6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:56.129508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb7f1c-4049-4ebe-a6ed-1cf6b9546f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:56.130126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb7f1c-4049-4ebe-a6ed-1cf6b9546f0e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:56.130403 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb7f1c-4049-4ebe-a6ed-1cf6b9546f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:56.130677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb7f1c-4049-4ebe-a6ed-1cf6b9546f0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:56.140795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb7f1c-4049-4ebe-a6ed-1cf6b9546f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d7cb7f1c-4049-4ebe-a6ed-1cf6b9546f0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:47:56.146127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:47:56.147213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:47:56.148477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 558/1101] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:47:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:47:56.153145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0f6e6-69a3-4314-b4a5-bfb161d3a42f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:56.155496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:56.252313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0f6e6-69a3-4314-b4a5-bfb161d3a42f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:56.287973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041a54b-4c33-44e4-94c6-267294a5784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:56.288591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041a54b-4c33-44e4-94c6-267294a5784f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:56.288869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041a54b-4c33-44e4-94c6-267294a5784f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:56.289144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041a54b-4c33-44e4-94c6-267294a5784f None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:56.289853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041a54b-4c33-44e4-94c6-267294a5784f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:56.290155 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041a54b-4c33-44e4-94c6-267294a5784f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:56.291539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041a54b-4c33-44e4-94c6-267294a5784f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:56.299728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 544/1102] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:47:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:56.307051 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb7f1c-4049-4ebe-a6ed-1cf6b9546f0e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:56.408850 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb7f1c-4049-4ebe-a6ed-1cf6b9546f0e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:56.442361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18987f49-962c-4a41-8b88-8a0795a476f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:56.443017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18987f49-962c-4a41-8b88-8a0795a476f4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:56.443300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18987f49-962c-4a41-8b88-8a0795a476f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:56.443576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18987f49-962c-4a41-8b88-8a0795a476f4 None admin] PATH_INFO: `/v3/users/bade6f2784e547afac484a0d4fbda596` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:56.444742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18987f49-962c-4a41-8b88-8a0795a476f4 None admin] RBAC: Authorizing `identity:get_user(user_id=bade6f2784e547afac484a0d4fbda596)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:56.446169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18987f49-962c-4a41-8b88-8a0795a476f4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:56.447825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 559/1103] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:47:56 2022] GET /identity/v3/users/bade6f2784e547afac484a0d4fbda596 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:56.453569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041a54b-4c33-44e4-94c6-267294a5784f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:56.514561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:56.553825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041a54b-4c33-44e4-94c6-267294a5784f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:56.587168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a96ade-69a7-4718-b0e8-32dc23497cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:56.587775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a96ade-69a7-4718-b0e8-32dc23497cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:56.588054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a96ade-69a7-4718-b0e8-32dc23497cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:56.588349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a96ade-69a7-4718-b0e8-32dc23497cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:56.652486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a96ade-69a7-4718-b0e8-32dc23497cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:56.653908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a96ade-69a7-4718-b0e8-32dc23497cb7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:56.711220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:56.747874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 545/1104] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:47:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 03:47:56.753743 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18987f49-962c-4a41-8b88-8a0795a476f4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:56.769765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:56.850392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18987f49-962c-4a41-8b88-8a0795a476f4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:56.881989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd0ff8-547a-4699-8719-e82aff0a4c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:56.882606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd0ff8-547a-4699-8719-e82aff0a4c87 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:56.882913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd0ff8-547a-4699-8719-e82aff0a4c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:56.883188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd0ff8-547a-4699-8719-e82aff0a4c87 None admin] PATH_INFO: `/v3/projects/0f2ee9ffc86f418680d23d388b3543d9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:56.884324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd0ff8-547a-4699-8719-e82aff0a4c87 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2ee9ffc86f418680d23d388b3543d9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:56.885737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd0ff8-547a-4699-8719-e82aff0a4c87 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:56.887387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 560/1105] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:47:56 2022] GET /identity/v3/projects/0f2ee9ffc86f418680d23d388b3543d9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:56.893929 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a96ade-69a7-4718-b0e8-32dc23497cb7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:56.989348 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a96ade-69a7-4718-b0e8-32dc23497cb7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:57.022627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe7bd0-3995-45e2-8087-5487716e519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:57.023260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe7bd0-3995-45e2-8087-5487716e519d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:57.023539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe7bd0-3995-45e2-8087-5487716e519d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:57.023810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe7bd0-3995-45e2-8087-5487716e519d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:57.024577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe7bd0-3995-45e2-8087-5487716e519d None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=0f2ee9ffc86f418680d23d388b3543d9, user.id=bade6f2784e547afac484a0d4fbda596) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:47:57.024883 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe7bd0-3995-45e2-8087-5487716e519d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:57.026291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe7bd0-3995-45e2-8087-5487716e519d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:57.031914 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 546/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:47:56 2022] GET /identity/v3/role_assignments?user.id=bade6f2784e547afac484a0d4fbda596&scope.project.id=0f2ee9ffc86f418680d23d388b3543d9&role.id=a37ae24c516b48c69e13721c013134a2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:47:59.007235 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4bc5b-27b9-4c9e-b352-9094a87fda73 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:59.007787 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4bc5b-27b9-4c9e-b352-9094a87fda73 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:59.007787 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4bc5b-27b9-4c9e-b352-9094a87fda73 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:59.009133 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 561/1107] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:47:59.014080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de31d79-654d-4fd4-8b3a-b6edfc73061d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:59.014468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de31d79-654d-4fd4-8b3a-b6edfc73061d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:59.014623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de31d79-654d-4fd4-8b3a-b6edfc73061d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:59.032651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3de31d79-654d-4fd4-8b3a-b6edfc73061d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:47:59.034594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3de31d79-654d-4fd4-8b3a-b6edfc73061d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:47:59.057035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:59.100298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 547/1108] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:47:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:47:59.124311 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4bc5b-27b9-4c9e-b352-9094a87fda73 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:59.205294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:47:59.224992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4bc5b-27b9-4c9e-b352-9094a87fda73 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:59.257553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec4830-b813-4cb8-a0b1-8f15a784a505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:59.258124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec4830-b813-4cb8-a0b1-8f15a784a505 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:59.258407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec4830-b813-4cb8-a0b1-8f15a784a505 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:59.258688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec4830-b813-4cb8-a0b1-8f15a784a505 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:59.259859 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec4830-b813-4cb8-a0b1-8f15a784a505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:59.261284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec4830-b813-4cb8-a0b1-8f15a784a505 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:59.262885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 562/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:47:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:47:59.268541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de31d79-654d-4fd4-8b3a-b6edfc73061d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:47:59.369471 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de31d79-654d-4fd4-8b3a-b6edfc73061d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:47:59.407407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32429720-93ef-4d51-8c2d-7c263ba49843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:47:59.408546 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32429720-93ef-4d51-8c2d-7c263ba49843 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:47:59.408858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32429720-93ef-4d51-8c2d-7c263ba49843 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:47:59.409130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32429720-93ef-4d51-8c2d-7c263ba49843 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:47:59.409816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32429720-93ef-4d51-8c2d-7c263ba49843 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:47:59.411531 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32429720-93ef-4d51-8c2d-7c263ba49843 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:47:59.432627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 548/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:47:59 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:48:01.472375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bbf94-4c0c-4262-80e5-d360d788f078 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:01.473014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bbf94-4c0c-4262-80e5-d360d788f078 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:01.473014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bbf94-4c0c-4262-80e5-d360d788f078 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:01.474278 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 563/1111] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:01.479564 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7544403-7a82-433b-afe7-95fb63a1a910 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:01.480009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7544403-7a82-433b-afe7-95fb63a1a910 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:01.480108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7544403-7a82-433b-afe7-95fb63a1a910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:01.498515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b7544403-7a82-433b-afe7-95fb63a1a910 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:01.499741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b7544403-7a82-433b-afe7-95fb63a1a910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:01.509018 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:01.568359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 549/1112] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:01.590943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bbf94-4c0c-4262-80e5-d360d788f078 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:01.690891 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bbf94-4c0c-4262-80e5-d360d788f078 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:01.723507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e1077-5d4e-44e9-8741-6b03e7f45c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:01.724078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e1077-5d4e-44e9-8741-6b03e7f45c30 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:01.724362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e1077-5d4e-44e9-8741-6b03e7f45c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:01.724647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e1077-5d4e-44e9-8741-6b03e7f45c30 None admin] PATH_INFO: `/v3/roles/854a7f7c663b4c66a08f7e731c7bf63c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:01.726077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e1077-5d4e-44e9-8741-6b03e7f45c30 None admin] RBAC: Authorizing `identity:get_role(role_id=854a7f7c663b4c66a08f7e731c7bf63c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:01.727536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e1077-5d4e-44e9-8741-6b03e7f45c30 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:01.728863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 564/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:01 2022] GET /identity/v3/roles/854a7f7c663b4c66a08f7e731c7bf63c => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:01.734761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7544403-7a82-433b-afe7-95fb63a1a910 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:01.830917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7544403-7a82-433b-afe7-95fb63a1a910 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:01.863327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751c72b-aaa8-4d65-8920-59eaddb1eb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:01.863950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751c72b-aaa8-4d65-8920-59eaddb1eb40 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:01.864225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751c72b-aaa8-4d65-8920-59eaddb1eb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:01.864494 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751c72b-aaa8-4d65-8920-59eaddb1eb40 None admin] PATH_INFO: `/v3/users/884240eb3dd34649a2da8ffdb9c07432` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:01.877180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751c72b-aaa8-4d65-8920-59eaddb1eb40 None admin] RBAC: Authorizing `identity:get_user(user_id=884240eb3dd34649a2da8ffdb9c07432)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:01.878594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751c72b-aaa8-4d65-8920-59eaddb1eb40 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:01.880208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 550/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:01 2022] GET /identity/v3/users/884240eb3dd34649a2da8ffdb9c07432 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:01.885802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e1077-5d4e-44e9-8741-6b03e7f45c30 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:01.905048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:01.985258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e1077-5d4e-44e9-8741-6b03e7f45c30 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:02.019234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c98809-24d8-45e4-ada0-d4ba918fd1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:02.019827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c98809-24d8-45e4-ada0-d4ba918fd1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:02.020111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c98809-24d8-45e4-ada0-d4ba918fd1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:02.020381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c98809-24d8-45e4-ada0-d4ba918fd1f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:02.085391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c98809-24d8-45e4-ada0-d4ba918fd1f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:02.086846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c98809-24d8-45e4-ada0-d4ba918fd1f8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:02.104138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:02.182506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 565/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:48:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:48:02.188339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751c72b-aaa8-4d65-8920-59eaddb1eb40 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:02.191696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:02.290206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751c72b-aaa8-4d65-8920-59eaddb1eb40 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:02.322699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3761d0-b9d2-4658-abf1-9ac0a86d0a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:02.323354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3761d0-b9d2-4658-abf1-9ac0a86d0a7c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:02.323636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3761d0-b9d2-4658-abf1-9ac0a86d0a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:02.323909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3761d0-b9d2-4658-abf1-9ac0a86d0a7c None admin] PATH_INFO: `/v3/projects/0f2ee9ffc86f418680d23d388b3543d9` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:02.325045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3761d0-b9d2-4658-abf1-9ac0a86d0a7c None admin] RBAC: Authorizing `identity:get_project(project_id=0f2ee9ffc86f418680d23d388b3543d9)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:02.326467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3761d0-b9d2-4658-abf1-9ac0a86d0a7c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:02.328142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 551/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:48:02 2022] GET /identity/v3/projects/0f2ee9ffc86f418680d23d388b3543d9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:02.334920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c98809-24d8-45e4-ada0-d4ba918fd1f8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:02.430318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c98809-24d8-45e4-ada0-d4ba918fd1f8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:02.447464 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:02.464922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec655f2-8c1a-4548-a78f-1d3304da0ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:02.465529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec655f2-8c1a-4548-a78f-1d3304da0ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:02.465812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec655f2-8c1a-4548-a78f-1d3304da0ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:02.466089 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec655f2-8c1a-4548-a78f-1d3304da0ae3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:02.466897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec655f2-8c1a-4548-a78f-1d3304da0ae3 None admin] RBAC: Adding query filter params (role.id=854a7f7c663b4c66a08f7e731c7bf63c, scope.project.id=0f2ee9ffc86f418680d23d388b3543d9, user.id=884240eb3dd34649a2da8ffdb9c07432) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:02.467211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec655f2-8c1a-4548-a78f-1d3304da0ae3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:02.468632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec655f2-8c1a-4548-a78f-1d3304da0ae3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:02.474747 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 566/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:48:02 2022] GET /identity/v3/role_assignments?user.id=884240eb3dd34649a2da8ffdb9c07432&scope.project.id=0f2ee9ffc86f418680d23d388b3543d9&role.id=854a7f7c663b4c66a08f7e731c7bf63c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:04.445958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff7c4e-078f-4386-8130-7915c0336aee None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:04.446516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff7c4e-078f-4386-8130-7915c0336aee None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:04.446516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff7c4e-078f-4386-8130-7915c0336aee None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:04.448050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 552/1118] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:04.453177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1d89-c3e4-4aeb-8a20-dab546b3f87a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:04.453542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1d89-c3e4-4aeb-8a20-dab546b3f87a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:04.453734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1d89-c3e4-4aeb-8a20-dab546b3f87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:04.472781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-201f1d89-c3e4-4aeb-8a20-dab546b3f87a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:04.473956 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-201f1d89-c3e4-4aeb-8a20-dab546b3f87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:04.539247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 567/1119] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:04.561386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff7c4e-078f-4386-8130-7915c0336aee None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:04.630534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:04.662030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff7c4e-078f-4386-8130-7915c0336aee None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:04.694635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080b44-f381-46c3-ae89-65a53c8bf218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:04.695290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080b44-f381-46c3-ae89-65a53c8bf218 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:04.695571 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080b44-f381-46c3-ae89-65a53c8bf218 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:04.695846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080b44-f381-46c3-ae89-65a53c8bf218 None admin] PATH_INFO: `/v3/roles/854a7f7c663b4c66a08f7e731c7bf63c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:04.697319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa080b44-f381-46c3-ae89-65a53c8bf218 None admin] RBAC: Authorizing `identity:get_role(role_id=854a7f7c663b4c66a08f7e731c7bf63c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:04.698630 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa080b44-f381-46c3-ae89-65a53c8bf218 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:04.699989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 553/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:04 2022] GET /identity/v3/roles/854a7f7c663b4c66a08f7e731c7bf63c => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:04.706386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f1d89-c3e4-4aeb-8a20-dab546b3f87a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:04.803077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f1d89-c3e4-4aeb-8a20-dab546b3f87a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:04.815749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:04.837201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9825a-bc38-4e93-a040-353004c26dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:04.837833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9825a-bc38-4e93-a040-353004c26dbf None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:04.838116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9825a-bc38-4e93-a040-353004c26dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:04.838394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9825a-bc38-4e93-a040-353004c26dbf None admin] PATH_INFO: `/v3/users/884240eb3dd34649a2da8ffdb9c07432` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:04.839595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9825a-bc38-4e93-a040-353004c26dbf None admin] RBAC: Authorizing `identity:get_user(user_id=884240eb3dd34649a2da8ffdb9c07432)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:04.841031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9825a-bc38-4e93-a040-353004c26dbf None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:04.842625 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 568/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:04 2022] GET /identity/v3/users/884240eb3dd34649a2da8ffdb9c07432 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:04.848984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080b44-f381-46c3-ae89-65a53c8bf218 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:04.946771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080b44-f381-46c3-ae89-65a53c8bf218 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:04.979534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adba10d-3760-4ba1-94a4-2638225d8c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:04.980110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adba10d-3760-4ba1-94a4-2638225d8c23 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:04.980391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adba10d-3760-4ba1-94a4-2638225d8c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:04.980663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adba10d-3760-4ba1-94a4-2638225d8c23 None admin] PATH_INFO: `/v3/projects/0f2ee9ffc86f418680d23d388b3543d9` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:04.981813 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adba10d-3760-4ba1-94a4-2638225d8c23 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2ee9ffc86f418680d23d388b3543d9)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:04.983267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adba10d-3760-4ba1-94a4-2638225d8c23 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:04.984908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 554/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:48:04 2022] GET /identity/v3/projects/0f2ee9ffc86f418680d23d388b3543d9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:04.991447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9825a-bc38-4e93-a040-353004c26dbf None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:05.087056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9825a-bc38-4e93-a040-353004c26dbf None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:05.119371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6df03-c65b-4412-8146-70f654046440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:05.120019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6df03-c65b-4412-8146-70f654046440 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:05.120305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6df03-c65b-4412-8146-70f654046440 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:05.120578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6df03-c65b-4412-8146-70f654046440 None admin] PATH_INFO: `/v3/projects/0f2ee9ffc86f418680d23d388b3543d9/users/884240eb3dd34649a2da8ffdb9c07432/roles/854a7f7c663b4c66a08f7e731c7bf63c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:05.122694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6df03-c65b-4412-8146-70f654046440 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f2ee9ffc86f418680d23d388b3543d9, user_id=884240eb3dd34649a2da8ffdb9c07432, role_id=854a7f7c663b4c66a08f7e731c7bf63c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:05.124123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6df03-c65b-4412-8146-70f654046440 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:05.137362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 569/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:48:04 2022] PUT /identity/v3/projects/0f2ee9ffc86f418680d23d388b3543d9/users/884240eb3dd34649a2da8ffdb9c07432/roles/854a7f7c663b4c66a08f7e731c7bf63c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 03:48:07.140487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209f787-fb60-45e1-b432-8a49e284a1ab None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:07.141035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209f787-fb60-45e1-b432-8a49e284a1ab None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:07.141035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209f787-fb60-45e1-b432-8a49e284a1ab None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:07.142889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 555/1124] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:07.147941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd72a5-f140-4aad-9480-0c641cdf1b74 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:07.148325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd72a5-f140-4aad-9480-0c641cdf1b74 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:07.148503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd72a5-f140-4aad-9480-0c641cdf1b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:07.167983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c8dd72a5-f140-4aad-9480-0c641cdf1b74 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:07.169173 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c8dd72a5-f140-4aad-9480-0c641cdf1b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:07.202112 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:07.235574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 570/1125] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:07.258989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209f787-fb60-45e1-b432-8a49e284a1ab None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:07.268422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:07.356966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209f787-fb60-45e1-b432-8a49e284a1ab None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:07.388576 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ef286-563b-474c-9cbb-266692b339a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:07.389192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ef286-563b-474c-9cbb-266692b339a2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:07.389471 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ef286-563b-474c-9cbb-266692b339a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:07.389741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ef286-563b-474c-9cbb-266692b339a2 None admin] PATH_INFO: `/v3/roles/854a7f7c663b4c66a08f7e731c7bf63c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:07.391235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ef286-563b-474c-9cbb-266692b339a2 None admin] RBAC: Authorizing `identity:get_role(role_id=854a7f7c663b4c66a08f7e731c7bf63c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:07.393145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ef286-563b-474c-9cbb-266692b339a2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:07.394464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 556/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:07 2022] GET /identity/v3/roles/854a7f7c663b4c66a08f7e731c7bf63c => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:07.400280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd72a5-f140-4aad-9480-0c641cdf1b74 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:07.498211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd72a5-f140-4aad-9480-0c641cdf1b74 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:07.531754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc31b6-c382-4641-b6e1-828c6ebab66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:07.532414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc31b6-c382-4641-b6e1-828c6ebab66d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:07.532700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc31b6-c382-4641-b6e1-828c6ebab66d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:07.533011 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc31b6-c382-4641-b6e1-828c6ebab66d None admin] PATH_INFO: `/v3/users/884240eb3dd34649a2da8ffdb9c07432` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:07.534237 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc31b6-c382-4641-b6e1-828c6ebab66d None admin] RBAC: Authorizing `identity:get_user(user_id=884240eb3dd34649a2da8ffdb9c07432)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:07.537789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc31b6-c382-4641-b6e1-828c6ebab66d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:07.537789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 571/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:07 2022] GET /identity/v3/users/884240eb3dd34649a2da8ffdb9c07432 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:07.543385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ef286-563b-474c-9cbb-266692b339a2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:07.615777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:07.640333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ef286-563b-474c-9cbb-266692b339a2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:07.674170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac1a38-a5e3-4301-85ff-fffa15e91e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:07.674704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac1a38-a5e3-4301-85ff-fffa15e91e6d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:07.675013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac1a38-a5e3-4301-85ff-fffa15e91e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:07.675283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac1a38-a5e3-4301-85ff-fffa15e91e6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:07.738393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac1a38-a5e3-4301-85ff-fffa15e91e6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:07.739851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac1a38-a5e3-4301-85ff-fffa15e91e6d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:07.809303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:07.834502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 557/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:48:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:48:07.840482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc31b6-c382-4641-b6e1-828c6ebab66d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:07.879537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:07.940873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc31b6-c382-4641-b6e1-828c6ebab66d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:07.974157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90244d-5c3e-47bc-85ca-3a1bdf7ea8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:07.974815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90244d-5c3e-47bc-85ca-3a1bdf7ea8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:07.975097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90244d-5c3e-47bc-85ca-3a1bdf7ea8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:07.975370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90244d-5c3e-47bc-85ca-3a1bdf7ea8c2 None admin] PATH_INFO: `/v3/projects/0f2ee9ffc86f418680d23d388b3543d9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:07.976519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90244d-5c3e-47bc-85ca-3a1bdf7ea8c2 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2ee9ffc86f418680d23d388b3543d9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:07.977942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90244d-5c3e-47bc-85ca-3a1bdf7ea8c2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:07.979623 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 572/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:48:07 2022] GET /identity/v3/projects/0f2ee9ffc86f418680d23d388b3543d9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:07.985743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac1a38-a5e3-4301-85ff-fffa15e91e6d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:08.080121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac1a38-a5e3-4301-85ff-fffa15e91e6d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:08.112817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1923e17-0b12-4804-896a-a9c9277fac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:08.113421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1923e17-0b12-4804-896a-a9c9277fac6d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:08.113704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1923e17-0b12-4804-896a-a9c9277fac6d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:08.113979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1923e17-0b12-4804-896a-a9c9277fac6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:08.114758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1923e17-0b12-4804-896a-a9c9277fac6d None admin] RBAC: Adding query filter params (role.id=854a7f7c663b4c66a08f7e731c7bf63c, scope.project.id=0f2ee9ffc86f418680d23d388b3543d9, user.id=884240eb3dd34649a2da8ffdb9c07432) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:08.115099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1923e17-0b12-4804-896a-a9c9277fac6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:08.116513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1923e17-0b12-4804-896a-a9c9277fac6d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:08.122148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 558/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:48:07 2022] GET /identity/v3/role_assignments?user.id=884240eb3dd34649a2da8ffdb9c07432&scope.project.id=0f2ee9ffc86f418680d23d388b3543d9&role.id=854a7f7c663b4c66a08f7e731c7bf63c => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:10.122146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c027-ebb8-4327-a706-9b6203f61491 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:10.122682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c027-ebb8-4327-a706-9b6203f61491 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:10.123089 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c027-ebb8-4327-a706-9b6203f61491 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:10.125789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 573/1131] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:10.131466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee47f57-e85c-4c81-b7e3-e0e5fd9e10e2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:10.131892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee47f57-e85c-4c81-b7e3-e0e5fd9e10e2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:10.132027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee47f57-e85c-4c81-b7e3-e0e5fd9e10e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:10.151498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fee47f57-e85c-4c81-b7e3-e0e5fd9e10e2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:10.152707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fee47f57-e85c-4c81-b7e3-e0e5fd9e10e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:10.184613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:10.229820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 559/1132] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:10.254303 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7c027-ebb8-4327-a706-9b6203f61491 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:10.394592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7c027-ebb8-4327-a706-9b6203f61491 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:10.405242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:10.439868 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25262a9-172e-4648-9d38-583199d5c3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:10.440606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25262a9-172e-4648-9d38-583199d5c3ae None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:10.440887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25262a9-172e-4648-9d38-583199d5c3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:10.441164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25262a9-172e-4648-9d38-583199d5c3ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:10.442313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25262a9-172e-4648-9d38-583199d5c3ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:10.444193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25262a9-172e-4648-9d38-583199d5c3ae None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:10.446034 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 574/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:48:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:10.452332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee47f57-e85c-4c81-b7e3-e0e5fd9e10e2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:10.559976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee47f57-e85c-4c81-b7e3-e0e5fd9e10e2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:10.599958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63620aa7-504e-4223-84e3-305e49966707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:10.600496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620aa7-504e-4223-84e3-305e49966707 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:10.600667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620aa7-504e-4223-84e3-305e49966707 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:10.600935 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620aa7-504e-4223-84e3-305e49966707 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:10.601778 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63620aa7-504e-4223-84e3-305e49966707 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:10.604460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63620aa7-504e-4223-84e3-305e49966707 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:10.641130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 560/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:48:10 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:48:12.708446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf00b2-e8ad-418b-8390-c821183455a8 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:12.708994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf00b2-e8ad-418b-8390-c821183455a8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:12.708994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf00b2-e8ad-418b-8390-c821183455a8 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:12.710334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 575/1135] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:48:12.715601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea579396-e090-4b56-a849-94bcd0c6af6d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:12.716014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea579396-e090-4b56-a849-94bcd0c6af6d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:12.716150 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea579396-e090-4b56-a849-94bcd0c6af6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:12.734202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ea579396-e090-4b56-a849-94bcd0c6af6d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:12.735530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ea579396-e090-4b56-a849-94bcd0c6af6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:12.749108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:12.801765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 561/1136] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:12.826612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf00b2-e8ad-418b-8390-c821183455a8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:12.925755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf00b2-e8ad-418b-8390-c821183455a8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:12.958997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747eb0ed-cf36-4bd0-b971-44517cf4b48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:12.959620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eb0ed-cf36-4bd0-b971-44517cf4b48b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:12.959899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eb0ed-cf36-4bd0-b971-44517cf4b48b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:12.960174 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eb0ed-cf36-4bd0-b971-44517cf4b48b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:12.961313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747eb0ed-cf36-4bd0-b971-44517cf4b48b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:12.962731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747eb0ed-cf36-4bd0-b971-44517cf4b48b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:12.964323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 576/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:48:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:12.970021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea579396-e090-4b56-a849-94bcd0c6af6d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:13.067255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea579396-e090-4b56-a849-94bcd0c6af6d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:13.099346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36219738-d440-483f-bcfb-8f663223ef2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:13.099948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36219738-d440-483f-bcfb-8f663223ef2d None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:13.100231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36219738-d440-483f-bcfb-8f663223ef2d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:13.100505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36219738-d440-483f-bcfb-8f663223ef2d None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:13.101179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36219738-d440-483f-bcfb-8f663223ef2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:13.102599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36219738-d440-483f-bcfb-8f663223ef2d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:13.123177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 562/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:48:12 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:48:15.139758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663028e2-85e4-4179-a274-d9775808e9ec None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:15.140332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663028e2-85e4-4179-a274-d9775808e9ec None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:15.140332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663028e2-85e4-4179-a274-d9775808e9ec None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:15.141673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 577/1139] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:15.147238 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd135435-079f-4ea7-a3e6-e7a2cdb55210 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:15.147631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd135435-079f-4ea7-a3e6-e7a2cdb55210 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:15.147787 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd135435-079f-4ea7-a3e6-e7a2cdb55210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:15.160586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:15.166527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cd135435-079f-4ea7-a3e6-e7a2cdb55210 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:15.168303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cd135435-079f-4ea7-a3e6-e7a2cdb55210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:15.232571 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 563/1140] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:15.255532 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663028e2-85e4-4179-a274-d9775808e9ec None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:15.300096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:15.356124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663028e2-85e4-4179-a274-d9775808e9ec None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:15.388487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dedf2-9e0f-4149-9be8-ba5950ae9a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:15.389111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dedf2-9e0f-4149-9be8-ba5950ae9a6c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:15.389396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dedf2-9e0f-4149-9be8-ba5950ae9a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:15.389674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dedf2-9e0f-4149-9be8-ba5950ae9a6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:15.400318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dedf2-9e0f-4149-9be8-ba5950ae9a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-8d7dedf2-9e0f-4149-9be8-ba5950ae9a6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:48:15.405106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:48:15.406247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:48:15.407824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 578/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:48:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:48:15.412719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd135435-079f-4ea7-a3e6-e7a2cdb55210 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:15.513509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd135435-079f-4ea7-a3e6-e7a2cdb55210 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:15.546329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d932c-5e4b-49d6-981b-da1dfd909d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:15.546980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d932c-5e4b-49d6-981b-da1dfd909d09 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:15.547269 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d932c-5e4b-49d6-981b-da1dfd909d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:15.547547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d932c-5e4b-49d6-981b-da1dfd909d09 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:15.548823 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d932c-5e4b-49d6-981b-da1dfd909d09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:15.549158 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d932c-5e4b-49d6-981b-da1dfd909d09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:15.550537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d932c-5e4b-49d6-981b-da1dfd909d09 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:15.552835 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:15.559584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 564/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:48:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:15.566122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dedf2-9e0f-4149-9be8-ba5950ae9a6c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:15.664753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dedf2-9e0f-4149-9be8-ba5950ae9a6c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:15.666160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:15.697665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc445d02-dcab-4d28-abb8-9c6781d00b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:15.698285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc445d02-dcab-4d28-abb8-9c6781d00b4c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:15.698568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc445d02-dcab-4d28-abb8-9c6781d00b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:15.698870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc445d02-dcab-4d28-abb8-9c6781d00b4c None admin] PATH_INFO: `/v3/users/0ca9c6ecfc30425a9de0708d5a9a5605` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:15.712065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc445d02-dcab-4d28-abb8-9c6781d00b4c None admin] RBAC: Authorizing `identity:get_user(user_id=0ca9c6ecfc30425a9de0708d5a9a5605)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:15.713487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc445d02-dcab-4d28-abb8-9c6781d00b4c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:15.715125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 579/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:15 2022] GET /identity/v3/users/0ca9c6ecfc30425a9de0708d5a9a5605 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:15.721051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d932c-5e4b-49d6-981b-da1dfd909d09 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:15.819239 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d932c-5e4b-49d6-981b-da1dfd909d09 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:15.852091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1016b8-1d6f-4d4b-89aa-8e06fc31dc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:15.852701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1016b8-1d6f-4d4b-89aa-8e06fc31dc7c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:15.852987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1016b8-1d6f-4d4b-89aa-8e06fc31dc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:15.853264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1016b8-1d6f-4d4b-89aa-8e06fc31dc7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:15.916022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:15.920028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1016b8-1d6f-4d4b-89aa-8e06fc31dc7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:15.921471 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1016b8-1d6f-4d4b-89aa-8e06fc31dc7c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:16.017172 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 565/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:48:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:48:16.023015 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc445d02-dcab-4d28-abb8-9c6781d00b4c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:16.118329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc445d02-dcab-4d28-abb8-9c6781d00b4c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:16.152168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08e48e-6b03-47c4-9985-90187b633963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:16.152814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08e48e-6b03-47c4-9985-90187b633963 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:16.153056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08e48e-6b03-47c4-9985-90187b633963 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:16.153329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08e48e-6b03-47c4-9985-90187b633963 None admin] PATH_INFO: `/v3/projects/e2f2031bb02f45ec92f403b27da9e6e4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:16.154463 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08e48e-6b03-47c4-9985-90187b633963 None admin] RBAC: Authorizing `identity:get_project(project_id=e2f2031bb02f45ec92f403b27da9e6e4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:16.155919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08e48e-6b03-47c4-9985-90187b633963 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:16.157573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 580/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:48:16 2022] GET /identity/v3/projects/e2f2031bb02f45ec92f403b27da9e6e4 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:16.163859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1016b8-1d6f-4d4b-89aa-8e06fc31dc7c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:16.256505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:16.260278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1016b8-1d6f-4d4b-89aa-8e06fc31dc7c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:16.294178 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c9b56-4072-48fd-be98-6b80749c1bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:16.294810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9b56-4072-48fd-be98-6b80749c1bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:16.295093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9b56-4072-48fd-be98-6b80749c1bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:16.295367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9b56-4072-48fd-be98-6b80749c1bc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:16.296143 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c9b56-4072-48fd-be98-6b80749c1bc2 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=e2f2031bb02f45ec92f403b27da9e6e4, user.id=0ca9c6ecfc30425a9de0708d5a9a5605) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:16.296452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c9b56-4072-48fd-be98-6b80749c1bc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:16.298464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c9b56-4072-48fd-be98-6b80749c1bc2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:16.303701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 566/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:48:16 2022] GET /identity/v3/role_assignments?user.id=0ca9c6ecfc30425a9de0708d5a9a5605&scope.project.id=e2f2031bb02f45ec92f403b27da9e6e4&role.id=a37ae24c516b48c69e13721c013134a2 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:18.266021 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e924ee-26e3-49b7-8732-41de766adc8d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:18.266610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e924ee-26e3-49b7-8732-41de766adc8d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:18.266610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e924ee-26e3-49b7-8732-41de766adc8d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:18.267987 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 581/1147] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:18.273839 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9969229-9e8f-4105-bed1-4f7f0eb78b7c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:18.274212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9969229-9e8f-4105-bed1-4f7f0eb78b7c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:18.274390 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9969229-9e8f-4105-bed1-4f7f0eb78b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:18.292238 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c9969229-9e8f-4105-bed1-4f7f0eb78b7c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:18.293416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c9969229-9e8f-4105-bed1-4f7f0eb78b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:18.358125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 567/1148] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:18.380300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e924ee-26e3-49b7-8732-41de766adc8d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:18.408108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:18.478796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e924ee-26e3-49b7-8732-41de766adc8d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:18.510517 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a42b1e-7ac9-4198-9da1-4b6f244bc6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:18.511163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a42b1e-7ac9-4198-9da1-4b6f244bc6dc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:18.511445 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a42b1e-7ac9-4198-9da1-4b6f244bc6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:18.511718 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a42b1e-7ac9-4198-9da1-4b6f244bc6dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:18.522179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a42b1e-7ac9-4198-9da1-4b6f244bc6dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-b6a42b1e-7ac9-4198-9da1-4b6f244bc6dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:48:18.526892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:48:18.527985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:48:18.528879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 582/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:48:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:48:18.534739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9969229-9e8f-4105-bed1-4f7f0eb78b7c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:18.621187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:18.630626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9969229-9e8f-4105-bed1-4f7f0eb78b7c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:18.663281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a402f1-3081-4f81-9c07-4c84eee82746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:18.663879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a402f1-3081-4f81-9c07-4c84eee82746 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:18.664158 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a402f1-3081-4f81-9c07-4c84eee82746 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:18.664429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a402f1-3081-4f81-9c07-4c84eee82746 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:18.665128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a402f1-3081-4f81-9c07-4c84eee82746 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:18.665428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a402f1-3081-4f81-9c07-4c84eee82746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:18.666776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a402f1-3081-4f81-9c07-4c84eee82746 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:18.675056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 568/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:48:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:18.680948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a42b1e-7ac9-4198-9da1-4b6f244bc6dc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:18.765157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:18.777021 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a42b1e-7ac9-4198-9da1-4b6f244bc6dc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:18.810134 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059c842-d0bb-4e34-b2b9-6cd85bbd371a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:18.810755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059c842-d0bb-4e34-b2b9-6cd85bbd371a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:18.811067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059c842-d0bb-4e34-b2b9-6cd85bbd371a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:18.811340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059c842-d0bb-4e34-b2b9-6cd85bbd371a None admin] PATH_INFO: `/v3/users/0ca9c6ecfc30425a9de0708d5a9a5605` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:18.812479 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059c842-d0bb-4e34-b2b9-6cd85bbd371a None admin] RBAC: Authorizing `identity:get_user(user_id=0ca9c6ecfc30425a9de0708d5a9a5605)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:18.814027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059c842-d0bb-4e34-b2b9-6cd85bbd371a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:18.815616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 583/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:18 2022] GET /identity/v3/users/0ca9c6ecfc30425a9de0708d5a9a5605 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:18.821564 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a402f1-3081-4f81-9c07-4c84eee82746 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:18.924539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a402f1-3081-4f81-9c07-4c84eee82746 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:18.956409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5475b6-0aec-46b5-a67b-7b4e9bfb5ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:18.957034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5475b6-0aec-46b5-a67b-7b4e9bfb5ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:18.957318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5475b6-0aec-46b5-a67b-7b4e9bfb5ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:18.957591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5475b6-0aec-46b5-a67b-7b4e9bfb5ee7 None admin] PATH_INFO: `/v3/projects/e2f2031bb02f45ec92f403b27da9e6e4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:18.958727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5475b6-0aec-46b5-a67b-7b4e9bfb5ee7 None admin] RBAC: Authorizing `identity:get_project(project_id=e2f2031bb02f45ec92f403b27da9e6e4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:18.960173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5475b6-0aec-46b5-a67b-7b4e9bfb5ee7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:18.961802 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 569/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:48:18 2022] GET /identity/v3/projects/e2f2031bb02f45ec92f403b27da9e6e4 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:18.967844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059c842-d0bb-4e34-b2b9-6cd85bbd371a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:19.064486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059c842-d0bb-4e34-b2b9-6cd85bbd371a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:19.097312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c16599-28d5-4780-b831-3c46df18530f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:19.097955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c16599-28d5-4780-b831-3c46df18530f None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:19.098236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c16599-28d5-4780-b831-3c46df18530f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:19.098510 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c16599-28d5-4780-b831-3c46df18530f None admin] PATH_INFO: `/v3/projects/e2f2031bb02f45ec92f403b27da9e6e4/users/0ca9c6ecfc30425a9de0708d5a9a5605/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:19.100437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c16599-28d5-4780-b831-3c46df18530f None admin] RBAC: Authorizing `identity:create_grant(project_id=e2f2031bb02f45ec92f403b27da9e6e4, user_id=0ca9c6ecfc30425a9de0708d5a9a5605, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:19.101845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c16599-28d5-4780-b831-3c46df18530f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:19.114961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 584/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:48:18 2022] PUT /identity/v3/projects/e2f2031bb02f45ec92f403b27da9e6e4/users/0ca9c6ecfc30425a9de0708d5a9a5605/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 03:48:21.074922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f0086-c5f6-452f-97b1-94dc750c2c9d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:21.075477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f0086-c5f6-452f-97b1-94dc750c2c9d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:21.075477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f0086-c5f6-452f-97b1-94dc750c2c9d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:21.076835 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 570/1154] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:21.082806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b655d-d12b-41ae-ab98-5fdbe275ec18 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:21.083234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b655d-d12b-41ae-ab98-5fdbe275ec18 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:21.083365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b655d-d12b-41ae-ab98-5fdbe275ec18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:21.101761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-019b655d-d12b-41ae-ab98-5fdbe275ec18 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:21.102974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-019b655d-d12b-41ae-ab98-5fdbe275ec18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:21.113409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:21.170301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 585/1155] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:48:21.194017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f0086-c5f6-452f-97b1-94dc750c2c9d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:21.218296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:21.292093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f0086-c5f6-452f-97b1-94dc750c2c9d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:21.324244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cbed4-3c78-4ea1-bfdd-4c06a540e0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:21.324867 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cbed4-3c78-4ea1-bfdd-4c06a540e0ee None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:21.325198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cbed4-3c78-4ea1-bfdd-4c06a540e0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:21.325415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cbed4-3c78-4ea1-bfdd-4c06a540e0ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:21.336048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cbed4-3c78-4ea1-bfdd-4c06a540e0ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5b8cbed4-3c78-4ea1-bfdd-4c06a540e0ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:48:21.340800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:48:21.341878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:48:21.343015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 571/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:48:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:48:21.348589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b655d-d12b-41ae-ab98-5fdbe275ec18 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:21.442884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b655d-d12b-41ae-ab98-5fdbe275ec18 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:21.475324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaf866-65d2-486b-8bd0-f6bf7d93b641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:21.475925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaf866-65d2-486b-8bd0-f6bf7d93b641 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:21.476204 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaf866-65d2-486b-8bd0-f6bf7d93b641 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:21.476475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaf866-65d2-486b-8bd0-f6bf7d93b641 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:21.477178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacaf866-65d2-486b-8bd0-f6bf7d93b641 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:21.477478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacaf866-65d2-486b-8bd0-f6bf7d93b641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:21.478853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacaf866-65d2-486b-8bd0-f6bf7d93b641 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:21.487087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 586/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:48:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:21.493149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cbed4-3c78-4ea1-bfdd-4c06a540e0ee None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:21.574994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:21.593188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cbed4-3c78-4ea1-bfdd-4c06a540e0ee None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:21.627260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9fcc7-ed33-47ff-8217-440e31605718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:21.627895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9fcc7-ed33-47ff-8217-440e31605718 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:21.628177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9fcc7-ed33-47ff-8217-440e31605718 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:21.628453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9fcc7-ed33-47ff-8217-440e31605718 None admin] PATH_INFO: `/v3/users/0ca9c6ecfc30425a9de0708d5a9a5605` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:21.629602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9fcc7-ed33-47ff-8217-440e31605718 None admin] RBAC: Authorizing `identity:get_user(user_id=0ca9c6ecfc30425a9de0708d5a9a5605)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:21.631067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9fcc7-ed33-47ff-8217-440e31605718 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:21.632654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 572/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:21 2022] GET /identity/v3/users/0ca9c6ecfc30425a9de0708d5a9a5605 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:21.638373 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaf866-65d2-486b-8bd0-f6bf7d93b641 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:21.649435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:21.734758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaf866-65d2-486b-8bd0-f6bf7d93b641 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:21.767119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d369872-c22c-47a5-8f73-eec46e7e6237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:21.767720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d369872-c22c-47a5-8f73-eec46e7e6237 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:21.768004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d369872-c22c-47a5-8f73-eec46e7e6237 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:21.768282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d369872-c22c-47a5-8f73-eec46e7e6237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:21.832909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d369872-c22c-47a5-8f73-eec46e7e6237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:21.834334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d369872-c22c-47a5-8f73-eec46e7e6237 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:21.843525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:21.929543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 587/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:48:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:48:21.935459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9fcc7-ed33-47ff-8217-440e31605718 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:22.030902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9fcc7-ed33-47ff-8217-440e31605718 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:22.064393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdddef-f5a5-46c6-aba4-fe9005cb8f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:22.065021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdddef-f5a5-46c6-aba4-fe9005cb8f6a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:22.065299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdddef-f5a5-46c6-aba4-fe9005cb8f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:22.065572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdddef-f5a5-46c6-aba4-fe9005cb8f6a None admin] PATH_INFO: `/v3/projects/e2f2031bb02f45ec92f403b27da9e6e4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:22.066695 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdddef-f5a5-46c6-aba4-fe9005cb8f6a None admin] RBAC: Authorizing `identity:get_project(project_id=e2f2031bb02f45ec92f403b27da9e6e4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:22.068141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdddef-f5a5-46c6-aba4-fe9005cb8f6a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:22.069775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 573/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:48:21 2022] GET /identity/v3/projects/e2f2031bb02f45ec92f403b27da9e6e4 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:22.075876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d369872-c22c-47a5-8f73-eec46e7e6237 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:22.172487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d369872-c22c-47a5-8f73-eec46e7e6237 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:22.181922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:22.206495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcd402-004c-43f8-8358-fcb7d9016099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:22.207108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcd402-004c-43f8-8358-fcb7d9016099 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:22.207389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcd402-004c-43f8-8358-fcb7d9016099 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:22.207663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcd402-004c-43f8-8358-fcb7d9016099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:22.208444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcd402-004c-43f8-8358-fcb7d9016099 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=e2f2031bb02f45ec92f403b27da9e6e4, user.id=0ca9c6ecfc30425a9de0708d5a9a5605) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:22.208752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcd402-004c-43f8-8358-fcb7d9016099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:22.210164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcd402-004c-43f8-8358-fcb7d9016099 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:22.215750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 588/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:48:22 2022] GET /identity/v3/role_assignments?user.id=0ca9c6ecfc30425a9de0708d5a9a5605&scope.project.id=e2f2031bb02f45ec92f403b27da9e6e4&role.id=a37ae24c516b48c69e13721c013134a2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:24.195138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdd000-f9d9-4a50-9d4a-108d2a17aa55 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:24.195668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdd000-f9d9-4a50-9d4a-108d2a17aa55 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:24.195668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdd000-f9d9-4a50-9d4a-108d2a17aa55 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:24.197029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 574/1162] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:24.202004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cac09-6c06-4d47-b339-61310e34a487 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:24.202417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cac09-6c06-4d47-b339-61310e34a487 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:24.202555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cac09-6c06-4d47-b339-61310e34a487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:24.220953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-750cac09-6c06-4d47-b339-61310e34a487 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:24.222140 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-750cac09-6c06-4d47-b339-61310e34a487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:24.287159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 589/1163] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:24.308875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdd000-f9d9-4a50-9d4a-108d2a17aa55 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:24.327748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:24.408652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdd000-f9d9-4a50-9d4a-108d2a17aa55 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:24.440973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e44eb-23ab-490f-b6e6-f13204ba966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:24.441573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e44eb-23ab-490f-b6e6-f13204ba966b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:24.441851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e44eb-23ab-490f-b6e6-f13204ba966b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:24.442122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e44eb-23ab-490f-b6e6-f13204ba966b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:24.506219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e44eb-23ab-490f-b6e6-f13204ba966b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:24.507610 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e44eb-23ab-490f-b6e6-f13204ba966b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:24.524879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:24.607680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 575/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:48:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:48:24.613881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750cac09-6c06-4d47-b339-61310e34a487 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:24.709656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750cac09-6c06-4d47-b339-61310e34a487 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:24.711128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:24.742748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f1278-fc8b-4658-870c-067b8ecd6be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:24.743397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1278-fc8b-4658-870c-067b8ecd6be0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:24.743674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1278-fc8b-4658-870c-067b8ecd6be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:24.743945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1278-fc8b-4658-870c-067b8ecd6be0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:24.750893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f1278-fc8b-4658-870c-067b8ecd6be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:24.752502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f1278-fc8b-4658-870c-067b8ecd6be0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-c32f1278-fc8b-4658-870c-067b8ecd6be0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:48:24.763499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:48:24.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:48:24.771375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 590/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jul 28 03:48:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:48:24.772387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e44eb-23ab-490f-b6e6-f13204ba966b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:24.866202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e44eb-23ab-490f-b6e6-f13204ba966b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:24.883124 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:24.900379 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209f9db-f94f-4072-8657-4d688ef8f3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:24.900982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f9db-f94f-4072-8657-4d688ef8f3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:24.901266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f9db-f94f-4072-8657-4d688ef8f3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:24.901541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f9db-f94f-4072-8657-4d688ef8f3e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:24.902222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209f9db-f94f-4072-8657-4d688ef8f3e5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:24.902527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209f9db-f94f-4072-8657-4d688ef8f3e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:24.903910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209f9db-f94f-4072-8657-4d688ef8f3e5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:24.910850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 576/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jul 28 03:48:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:26.857053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd597c8b-62ea-4548-8e89-7a899ff1f47a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:26.857597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd597c8b-62ea-4548-8e89-7a899ff1f47a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:26.857597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd597c8b-62ea-4548-8e89-7a899ff1f47a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:26.858969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 591/1167] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:26.864149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107a9cf-02d6-4b82-af92-6a97e5aa9f38 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:26.864550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107a9cf-02d6-4b82-af92-6a97e5aa9f38 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:26.864702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107a9cf-02d6-4b82-af92-6a97e5aa9f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:26.882335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6107a9cf-02d6-4b82-af92-6a97e5aa9f38 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:26.883533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6107a9cf-02d6-4b82-af92-6a97e5aa9f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:26.947940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 577/1168] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:26.969915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd597c8b-62ea-4548-8e89-7a899ff1f47a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:27.068575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd597c8b-62ea-4548-8e89-7a899ff1f47a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:27.100855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afa200-7385-4b14-b083-d087466ca048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:27.101447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afa200-7385-4b14-b083-d087466ca048 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:27.101726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afa200-7385-4b14-b083-d087466ca048 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:27.101998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afa200-7385-4b14-b083-d087466ca048 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:27.102697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14afa200-7385-4b14-b083-d087466ca048 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:27.104150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14afa200-7385-4b14-b083-d087466ca048 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:27.119346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 592/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Thu Jul 28 03:48:26 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:48:29.099965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3c535-6391-46eb-a0eb-a46a2990c4bb None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:29.100517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3c535-6391-46eb-a0eb-a46a2990c4bb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:29.100517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3c535-6391-46eb-a0eb-a46a2990c4bb None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:29.101845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 578/1170] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:29.107022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7491db-6b6b-48f8-9c6c-faff9bd89221 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:29.107433 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7491db-6b6b-48f8-9c6c-faff9bd89221 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:29.107589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7491db-6b6b-48f8-9c6c-faff9bd89221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:29.125873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0d7491db-6b6b-48f8-9c6c-faff9bd89221 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:29.127672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0d7491db-6b6b-48f8-9c6c-faff9bd89221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:29.143583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:29.193980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 593/1171] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:29.216799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3c535-6391-46eb-a0eb-a46a2990c4bb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:29.315970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3c535-6391-46eb-a0eb-a46a2990c4bb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:29.328426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:29.349147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956a35d-f074-4aaa-98ff-9a0ec135514f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:29.349775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956a35d-f074-4aaa-98ff-9a0ec135514f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:29.350106 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956a35d-f074-4aaa-98ff-9a0ec135514f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:29.350338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956a35d-f074-4aaa-98ff-9a0ec135514f None admin] PATH_INFO: `/v3/domains/d9a05e67f0054c42b813d59de75808f4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:29.361020 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956a35d-f074-4aaa-98ff-9a0ec135514f None admin] RBAC: Authorizing `identity:get_domain(domain_id=d9a05e67f0054c42b813d59de75808f4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:29.362469 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956a35d-f074-4aaa-98ff-9a0ec135514f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:29.364091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 579/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Thu Jul 28 03:48:29 2022] GET /identity/v3/domains/d9a05e67f0054c42b813d59de75808f4 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:29.369857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7491db-6b6b-48f8-9c6c-faff9bd89221 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:29.464190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7491db-6b6b-48f8-9c6c-faff9bd89221 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:29.496981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30cb7e-492e-46c0-868c-c0eb0d8b90d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:29.497577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30cb7e-492e-46c0-868c-c0eb0d8b90d9 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:29.497856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30cb7e-492e-46c0-868c-c0eb0d8b90d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:29.498125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30cb7e-492e-46c0-868c-c0eb0d8b90d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:29.498815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30cb7e-492e-46c0-868c-c0eb0d8b90d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:29.500231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30cb7e-492e-46c0-868c-c0eb0d8b90d9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:29.534475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 594/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:48:29 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:48:31.579501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defd93f-ae86-4938-925d-1e928a27b9bd None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:31.579975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defd93f-ae86-4938-925d-1e928a27b9bd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:31.580123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defd93f-ae86-4938-925d-1e928a27b9bd None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:31.581480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 580/1174] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:31.586813 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acab00-f6e7-4237-81a7-53014bec291a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:31.587215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acab00-f6e7-4237-81a7-53014bec291a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:31.587381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acab00-f6e7-4237-81a7-53014bec291a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:31.594535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:31.606776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-33acab00-f6e7-4237-81a7-53014bec291a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:31.608680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-33acab00-f6e7-4237-81a7-53014bec291a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:31.673386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 595/1175] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:31.697096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defd93f-ae86-4938-925d-1e928a27b9bd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:31.795598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defd93f-ae86-4938-925d-1e928a27b9bd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:31.828312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcf575-c3c3-40c8-afad-a941ecc9f681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:31.828936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcf575-c3c3-40c8-afad-a941ecc9f681 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:31.829956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcf575-c3c3-40c8-afad-a941ecc9f681 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:31.830070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcf575-c3c3-40c8-afad-a941ecc9f681 None admin] PATH_INFO: `/v3/domains/d9a05e67f0054c42b813d59de75808f4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:31.831153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcf575-c3c3-40c8-afad-a941ecc9f681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d9a05e67f0054c42b813d59de75808f4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:31.832572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcf575-c3c3-40c8-afad-a941ecc9f681 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:31.834129 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 581/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Thu Jul 28 03:48:31 2022] GET /identity/v3/domains/d9a05e67f0054c42b813d59de75808f4 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:31.839806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acab00-f6e7-4237-81a7-53014bec291a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:31.936358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acab00-f6e7-4237-81a7-53014bec291a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:31.969255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08490e14-397c-492d-97fe-7b121dd7ce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:31.970507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08490e14-397c-492d-97fe-7b121dd7ce1f None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:31.970808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08490e14-397c-492d-97fe-7b121dd7ce1f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:31.971100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08490e14-397c-492d-97fe-7b121dd7ce1f None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:31.971779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08490e14-397c-492d-97fe-7b121dd7ce1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:31.973207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08490e14-397c-492d-97fe-7b121dd7ce1f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:31.981254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:31.995005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 596/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:48:31 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:48:34.008503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd2672-13be-4c0a-b2ac-3502512cb6b8 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:34.009090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd2672-13be-4c0a-b2ac-3502512cb6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:34.009090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd2672-13be-4c0a-b2ac-3502512cb6b8 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:34.010401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 582/1178] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:34.015401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18026ce-6006-407e-8f6b-e17a12fadbd6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:34.015796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18026ce-6006-407e-8f6b-e17a12fadbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:34.015955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18026ce-6006-407e-8f6b-e17a12fadbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:34.034103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b18026ce-6006-407e-8f6b-e17a12fadbd6 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:34.035301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b18026ce-6006-407e-8f6b-e17a12fadbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:34.099441 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 597/1179] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:34.122611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd2672-13be-4c0a-b2ac-3502512cb6b8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:34.166304 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:34.225216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd2672-13be-4c0a-b2ac-3502512cb6b8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:34.258274 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecf9db-1cd7-4c63-9930-627aca82ff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:34.258971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecf9db-1cd7-4c63-9930-627aca82ff9f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:34.259209 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecf9db-1cd7-4c63-9930-627aca82ff9f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:34.259483 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecf9db-1cd7-4c63-9930-627aca82ff9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:34.269240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecf9db-1cd7-4c63-9930-627aca82ff9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-43ecf9db-1cd7-4c63-9930-627aca82ff9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:48:34.274033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:48:34.275259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:48:34.276519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 583/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:48:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:48:34.281136 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18026ce-6006-407e-8f6b-e17a12fadbd6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:34.377100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18026ce-6006-407e-8f6b-e17a12fadbd6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:34.396387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:34.411406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adf1a0-71a2-4ccd-9f92-b5683b9e2d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:34.412020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adf1a0-71a2-4ccd-9f92-b5683b9e2d06 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:34.412305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adf1a0-71a2-4ccd-9f92-b5683b9e2d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:34.412582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adf1a0-71a2-4ccd-9f92-b5683b9e2d06 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:34.413298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adf1a0-71a2-4ccd-9f92-b5683b9e2d06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:34.413603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adf1a0-71a2-4ccd-9f92-b5683b9e2d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:34.414997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adf1a0-71a2-4ccd-9f92-b5683b9e2d06 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:34.423620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 598/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:48:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:34.429315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecf9db-1cd7-4c63-9930-627aca82ff9f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:34.524398 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:34.527290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecf9db-1cd7-4c63-9930-627aca82ff9f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:34.562418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c8ac3-41be-486e-9d9c-4a0c88f8c4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:34.563091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c8ac3-41be-486e-9d9c-4a0c88f8c4fe None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:34.563422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c8ac3-41be-486e-9d9c-4a0c88f8c4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:34.563705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c8ac3-41be-486e-9d9c-4a0c88f8c4fe None admin] PATH_INFO: `/v3/users/eb2e8c55739e4490aeb0efc4b64e2a40` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:34.576520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c8ac3-41be-486e-9d9c-4a0c88f8c4fe None admin] RBAC: Authorizing `identity:get_user(user_id=eb2e8c55739e4490aeb0efc4b64e2a40)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:34.577957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c8ac3-41be-486e-9d9c-4a0c88f8c4fe None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:34.579643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 584/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:34 2022] GET /identity/v3/users/eb2e8c55739e4490aeb0efc4b64e2a40 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:34.585358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adf1a0-71a2-4ccd-9f92-b5683b9e2d06 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:34.693262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adf1a0-71a2-4ccd-9f92-b5683b9e2d06 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:34.731674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2672a9-c7bb-472d-a0fb-6755385edfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:34.732306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2672a9-c7bb-472d-a0fb-6755385edfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:34.732592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2672a9-c7bb-472d-a0fb-6755385edfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:34.732872 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2672a9-c7bb-472d-a0fb-6755385edfc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:34.795818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:34.824151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2672a9-c7bb-472d-a0fb-6755385edfc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:34.828487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2672a9-c7bb-472d-a0fb-6755385edfc5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:34.943487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 599/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:48:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:48:34.949517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c8ac3-41be-486e-9d9c-4a0c88f8c4fe None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:35.047223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c8ac3-41be-486e-9d9c-4a0c88f8c4fe None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:35.081674 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d05b8e-3f9d-46a8-bc30-97ab45ed045b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:35.082300 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d05b8e-3f9d-46a8-bc30-97ab45ed045b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:35.082584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d05b8e-3f9d-46a8-bc30-97ab45ed045b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:35.082879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d05b8e-3f9d-46a8-bc30-97ab45ed045b None admin] PATH_INFO: `/v3/projects/313dd5aaf05141458d65539068f30785` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:35.084015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d05b8e-3f9d-46a8-bc30-97ab45ed045b None admin] RBAC: Authorizing `identity:get_project(project_id=313dd5aaf05141458d65539068f30785)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:35.085417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d05b8e-3f9d-46a8-bc30-97ab45ed045b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:35.087068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 585/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:48:34 2022] GET /identity/v3/projects/313dd5aaf05141458d65539068f30785 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:35.093116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2672a9-c7bb-472d-a0fb-6755385edfc5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:35.171502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:35.189697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2672a9-c7bb-472d-a0fb-6755385edfc5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:35.222750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e61b4-6fa7-4d3e-a325-0057f8e28595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:35.223378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e61b4-6fa7-4d3e-a325-0057f8e28595 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:35.223660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e61b4-6fa7-4d3e-a325-0057f8e28595 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:35.223935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e61b4-6fa7-4d3e-a325-0057f8e28595 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:35.224713 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e61b4-6fa7-4d3e-a325-0057f8e28595 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=313dd5aaf05141458d65539068f30785, user.id=eb2e8c55739e4490aeb0efc4b64e2a40) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:35.225021 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e61b4-6fa7-4d3e-a325-0057f8e28595 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:35.226462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e61b4-6fa7-4d3e-a325-0057f8e28595 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:35.231679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 600/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:48:35 2022] GET /identity/v3/role_assignments?user.id=eb2e8c55739e4490aeb0efc4b64e2a40&scope.project.id=313dd5aaf05141458d65539068f30785&role.id=a37ae24c516b48c69e13721c013134a2 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:37.174476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc5449-9ff8-477b-b792-ad7b8489468d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:37.175080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc5449-9ff8-477b-b792-ad7b8489468d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:37.175080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc5449-9ff8-477b-b792-ad7b8489468d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:37.176391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 586/1186] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:37.181753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36847414-d136-4e36-a587-6d37d4c6920d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:37.182135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36847414-d136-4e36-a587-6d37d4c6920d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:37.182310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36847414-d136-4e36-a587-6d37d4c6920d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:37.199885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-36847414-d136-4e36-a587-6d37d4c6920d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:37.201070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-36847414-d136-4e36-a587-6d37d4c6920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:37.265805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 601/1187] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:37.289737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc5449-9ff8-477b-b792-ad7b8489468d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:37.316420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:37.389073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc5449-9ff8-477b-b792-ad7b8489468d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:37.422202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209ff73-a30b-4cec-99d6-11ec9f95429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:37.423197 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209ff73-a30b-4cec-99d6-11ec9f95429d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:37.423496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209ff73-a30b-4cec-99d6-11ec9f95429d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:37.423773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209ff73-a30b-4cec-99d6-11ec9f95429d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:37.433541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209ff73-a30b-4cec-99d6-11ec9f95429d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-0209ff73-a30b-4cec-99d6-11ec9f95429d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:48:37.438908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:48:37.439997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:48:37.440882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 587/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:48:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:48:37.446282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36847414-d136-4e36-a587-6d37d4c6920d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:37.519759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:37.542669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36847414-d136-4e36-a587-6d37d4c6920d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:37.576317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f200e-7c1c-4149-8fe4-a0a15e9f52b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:37.576932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f200e-7c1c-4149-8fe4-a0a15e9f52b4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:37.577218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f200e-7c1c-4149-8fe4-a0a15e9f52b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:37.577493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f200e-7c1c-4149-8fe4-a0a15e9f52b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:37.578221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f200e-7c1c-4149-8fe4-a0a15e9f52b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:37.578556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f200e-7c1c-4149-8fe4-a0a15e9f52b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:37.580512 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f200e-7c1c-4149-8fe4-a0a15e9f52b4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:37.588546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 602/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:48:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:37.594495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209ff73-a30b-4cec-99d6-11ec9f95429d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:37.677934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:37.693243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209ff73-a30b-4cec-99d6-11ec9f95429d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:37.726219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a0270-6fe7-4936-a156-359dafee6db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:37.726938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a0270-6fe7-4936-a156-359dafee6db1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:37.727224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a0270-6fe7-4936-a156-359dafee6db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:37.727499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a0270-6fe7-4936-a156-359dafee6db1 None admin] PATH_INFO: `/v3/users/eb2e8c55739e4490aeb0efc4b64e2a40` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:37.728648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a0270-6fe7-4936-a156-359dafee6db1 None admin] RBAC: Authorizing `identity:get_user(user_id=eb2e8c55739e4490aeb0efc4b64e2a40)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:37.730067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a0270-6fe7-4936-a156-359dafee6db1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:37.731659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 588/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:37 2022] GET /identity/v3/users/eb2e8c55739e4490aeb0efc4b64e2a40 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:37.737639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f200e-7c1c-4149-8fe4-a0a15e9f52b4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:37.835910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f200e-7c1c-4149-8fe4-a0a15e9f52b4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:37.868629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bedec1-ecfd-4939-82d9-aeece56664ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:37.869256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bedec1-ecfd-4939-82d9-aeece56664ed None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:37.869540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bedec1-ecfd-4939-82d9-aeece56664ed None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:37.869819 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bedec1-ecfd-4939-82d9-aeece56664ed None admin] PATH_INFO: `/v3/projects/313dd5aaf05141458d65539068f30785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:37.870998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bedec1-ecfd-4939-82d9-aeece56664ed None admin] RBAC: Authorizing `identity:get_project(project_id=313dd5aaf05141458d65539068f30785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:37.872427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bedec1-ecfd-4939-82d9-aeece56664ed None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:37.874067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 603/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:48:37 2022] GET /identity/v3/projects/313dd5aaf05141458d65539068f30785 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:37.880667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a0270-6fe7-4936-a156-359dafee6db1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:37.975873 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a0270-6fe7-4936-a156-359dafee6db1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:38.009021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6566e-6449-4735-b972-c69543eac87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:38.009662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6566e-6449-4735-b972-c69543eac87f None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:38.009947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6566e-6449-4735-b972-c69543eac87f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:38.010216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6566e-6449-4735-b972-c69543eac87f None admin] PATH_INFO: `/v3/projects/313dd5aaf05141458d65539068f30785/users/eb2e8c55739e4490aeb0efc4b64e2a40/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:38.012133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6566e-6449-4735-b972-c69543eac87f None admin] RBAC: Authorizing `identity:create_grant(project_id=313dd5aaf05141458d65539068f30785, user_id=eb2e8c55739e4490aeb0efc4b64e2a40, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:38.013532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6566e-6449-4735-b972-c69543eac87f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:38.026319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 589/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:48:37 2022] PUT /identity/v3/projects/313dd5aaf05141458d65539068f30785/users/eb2e8c55739e4490aeb0efc4b64e2a40/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:48:39.978772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d24689-dcb8-4218-9e76-5baa25165dca None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:39.979351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d24689-dcb8-4218-9e76-5baa25165dca None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:39.979351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d24689-dcb8-4218-9e76-5baa25165dca None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:39.980696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 604/1193] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:48:39.985904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de689f-244b-49ed-8bbe-2ff2220f4aec None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:39.986328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de689f-244b-49ed-8bbe-2ff2220f4aec None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:39.986450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de689f-244b-49ed-8bbe-2ff2220f4aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:40.004512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-00de689f-244b-49ed-8bbe-2ff2220f4aec None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:48:40.005675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-00de689f-244b-49ed-8bbe-2ff2220f4aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:48:40.014528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:40.072561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 590/1194] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:48:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:48:40.095036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d24689-dcb8-4218-9e76-5baa25165dca None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:40.104125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:40.195556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d24689-dcb8-4218-9e76-5baa25165dca None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:40.227320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122cd98-dca1-4cfe-b736-541fc299ea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:40.227950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122cd98-dca1-4cfe-b736-541fc299ea00 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:40.228236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122cd98-dca1-4cfe-b736-541fc299ea00 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:40.228512 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122cd98-dca1-4cfe-b736-541fc299ea00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:40.239417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122cd98-dca1-4cfe-b736-541fc299ea00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-2122cd98-dca1-4cfe-b736-541fc299ea00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:48:40.244156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:48:40.245240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:48:40.246263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 605/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:48:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:48:40.251853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de689f-244b-49ed-8bbe-2ff2220f4aec None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:40.345339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de689f-244b-49ed-8bbe-2ff2220f4aec None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:40.378074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc61ca-eadc-4923-b083-9e2583c1a54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:40.378669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc61ca-eadc-4923-b083-9e2583c1a54d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:40.378982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc61ca-eadc-4923-b083-9e2583c1a54d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:40.379256 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc61ca-eadc-4923-b083-9e2583c1a54d None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:40.379960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc61ca-eadc-4923-b083-9e2583c1a54d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:40.380262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc61ca-eadc-4923-b083-9e2583c1a54d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:40.381619 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc61ca-eadc-4923-b083-9e2583c1a54d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:40.389304 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 591/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:48:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:40.395315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122cd98-dca1-4cfe-b736-541fc299ea00 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:40.458978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:40.493601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122cd98-dca1-4cfe-b736-541fc299ea00 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:40.525689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0033d5-9534-480b-8c28-4843c09f2673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:40.526320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0033d5-9534-480b-8c28-4843c09f2673 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:40.526605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0033d5-9534-480b-8c28-4843c09f2673 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:40.526908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0033d5-9534-480b-8c28-4843c09f2673 None admin] PATH_INFO: `/v3/users/eb2e8c55739e4490aeb0efc4b64e2a40` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:40.528061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0033d5-9534-480b-8c28-4843c09f2673 None admin] RBAC: Authorizing `identity:get_user(user_id=eb2e8c55739e4490aeb0efc4b64e2a40)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:40.529486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0033d5-9534-480b-8c28-4843c09f2673 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:40.531099 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 606/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jul 28 03:48:40 2022] GET /identity/v3/users/eb2e8c55739e4490aeb0efc4b64e2a40 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:40.537669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc61ca-eadc-4923-b083-9e2583c1a54d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:40.548109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:40.636308 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc61ca-eadc-4923-b083-9e2583c1a54d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:40.668987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fc150-1d18-45fe-8b06-3e944f1eca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:40.669587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fc150-1d18-45fe-8b06-3e944f1eca96 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:40.669867 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fc150-1d18-45fe-8b06-3e944f1eca96 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:40.670137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fc150-1d18-45fe-8b06-3e944f1eca96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:40.733952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157fc150-1d18-45fe-8b06-3e944f1eca96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:40.735476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157fc150-1d18-45fe-8b06-3e944f1eca96 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:40.742938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:40.831795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 592/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:48:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:48:40.837598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0033d5-9534-480b-8c28-4843c09f2673 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:40.934516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0033d5-9534-480b-8c28-4843c09f2673 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:40.966475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d19f79-62d7-4be0-8663-626cb3b24a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:40.967118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d19f79-62d7-4be0-8663-626cb3b24a78 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:40.967400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d19f79-62d7-4be0-8663-626cb3b24a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:40.967673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d19f79-62d7-4be0-8663-626cb3b24a78 None admin] PATH_INFO: `/v3/projects/313dd5aaf05141458d65539068f30785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:40.969132 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d19f79-62d7-4be0-8663-626cb3b24a78 None admin] RBAC: Authorizing `identity:get_project(project_id=313dd5aaf05141458d65539068f30785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:40.971768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d19f79-62d7-4be0-8663-626cb3b24a78 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:40.973395 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 607/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jul 28 03:48:40 2022] GET /identity/v3/projects/313dd5aaf05141458d65539068f30785 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:40.979387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fc150-1d18-45fe-8b06-3e944f1eca96 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:48:41.076453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fc150-1d18-45fe-8b06-3e944f1eca96 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:48:41.083834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:41.110590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca6607-69ac-4d9b-9b2d-f73db5d774b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:48:41.111232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6607-69ac-4d9b-9b2d-f73db5d774b3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:48:41.111516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6607-69ac-4d9b-9b2d-f73db5d774b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:48:41.111789 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6607-69ac-4d9b-9b2d-f73db5d774b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:48:41.112570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca6607-69ac-4d9b-9b2d-f73db5d774b3 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=313dd5aaf05141458d65539068f30785, user.id=eb2e8c55739e4490aeb0efc4b64e2a40) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:48:41.112884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca6607-69ac-4d9b-9b2d-f73db5d774b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:48:41.114303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca6607-69ac-4d9b-9b2d-f73db5d774b3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:48:41.120023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 593/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:48:40 2022] GET /identity/v3/role_assignments?user.id=eb2e8c55739e4490aeb0efc4b64e2a40&scope.project.id=313dd5aaf05141458d65539068f30785&role.id=a37ae24c516b48c69e13721c013134a2 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:48:50.962494 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:48:51.117384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=15 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:35.956036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8eb835-c792-4e9e-b1d3-699068b71530 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:35.956693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8eb835-c792-4e9e-b1d3-699068b71530 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:35.956693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8eb835-c792-4e9e-b1d3-699068b71530 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:35.958053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 608/1201] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:49:35.964066 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c11b6-5b73-4390-86f5-5adca91a624e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:35.964548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c11b6-5b73-4390-86f5-5adca91a624e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:35.964710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c11b6-5b73-4390-86f5-5adca91a624e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:35.985357 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e36c11b6-5b73-4390-86f5-5adca91a624e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:49:35.986560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e36c11b6-5b73-4390-86f5-5adca91a624e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:49:36.066836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 594/1202] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:49:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 03:49:36.090534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8eb835-c792-4e9e-b1d3-699068b71530 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:36.225005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8eb835-c792-4e9e-b1d3-699068b71530 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:36.282030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dde7e-ca96-4520-89b6-a4461ec89515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:36.283012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dde7e-ca96-4520-89b6-a4461ec89515 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:36.283292 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dde7e-ca96-4520-89b6-a4461ec89515 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:36.283564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dde7e-ca96-4520-89b6-a4461ec89515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:36.297640 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dde7e-ca96-4520-89b6-a4461ec89515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:36.299065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dde7e-ca96-4520-89b6-a4461ec89515 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-0c6dde7e-ca96-4520-89b6-a4461ec89515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:49:36.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:36.317049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:36.318076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:49:36.318076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:49:36.318076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:36.318076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:36.318076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 609/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:36.324851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c11b6-5b73-4390-86f5-5adca91a624e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:36.463025 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c11b6-5b73-4390-86f5-5adca91a624e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:36.502044 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:36.512348 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba88e7-4de2-42bf-bc20-d466d4dc2117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:36.513013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba88e7-4de2-42bf-bc20-d466d4dc2117 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:36.513298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba88e7-4de2-42bf-bc20-d466d4dc2117 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:36.514147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba88e7-4de2-42bf-bc20-d466d4dc2117 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:36.515068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba88e7-4de2-42bf-bc20-d466d4dc2117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:36.515376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba88e7-4de2-42bf-bc20-d466d4dc2117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:36.516810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba88e7-4de2-42bf-bc20-d466d4dc2117 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:36.533367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 595/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:36.539798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dde7e-ca96-4520-89b6-a4461ec89515 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:36.614624 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:36.680073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dde7e-ca96-4520-89b6-a4461ec89515 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:36.729000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e0640-8a5b-48a4-b88d-123af8c4c014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:36.729847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e0640-8a5b-48a4-b88d-123af8c4c014 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:36.730236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e0640-8a5b-48a4-b88d-123af8c4c014 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:36.730632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e0640-8a5b-48a4-b88d-123af8c4c014 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:36.731422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e0640-8a5b-48a4-b88d-123af8c4c014 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:36.736933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e0640-8a5b-48a4-b88d-123af8c4c014 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:36.760131 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 610/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jul 28 03:49:36 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:49:39.095463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815831b-293c-4946-b23e-156647e77780 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:39.096051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815831b-293c-4946-b23e-156647e77780 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:39.096051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815831b-293c-4946-b23e-156647e77780 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:39.097353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 596/1206] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:49:39.103207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30b9d1-968b-461b-bc61-08435df4afaa None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:39.103625 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30b9d1-968b-461b-bc61-08435df4afaa None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:39.103881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30b9d1-968b-461b-bc61-08435df4afaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:39.123331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7d30b9d1-968b-461b-bc61-08435df4afaa None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:49:39.124509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7d30b9d1-968b-461b-bc61-08435df4afaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:49:39.197178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 611/1207] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:49:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:49:39.223371 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815831b-293c-4946-b23e-156647e77780 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:39.356970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815831b-293c-4946-b23e-156647e77780 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:39.398110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c3e5a-c1e8-4d23-87b6-8fd411aa87dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:39.399216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c3e5a-c1e8-4d23-87b6-8fd411aa87dd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:39.399732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c3e5a-c1e8-4d23-87b6-8fd411aa87dd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:39.400206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c3e5a-c1e8-4d23-87b6-8fd411aa87dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:39.414198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c3e5a-c1e8-4d23-87b6-8fd411aa87dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6e6c3e5a-c1e8-4d23-87b6-8fd411aa87dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:49:39.421430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:39.422500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:39.423966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 597/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:49:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:39.429718 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30b9d1-968b-461b-bc61-08435df4afaa None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:39.485527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:39.564126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30b9d1-968b-461b-bc61-08435df4afaa None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:39.609066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e14a5-4449-485e-9c5c-5e6ebb43c1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:39.610237 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e14a5-4449-485e-9c5c-5e6ebb43c1fc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:39.610889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e14a5-4449-485e-9c5c-5e6ebb43c1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:39.611208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e14a5-4449-485e-9c5c-5e6ebb43c1fc None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:39.612115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e14a5-4449-485e-9c5c-5e6ebb43c1fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:39.612552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e14a5-4449-485e-9c5c-5e6ebb43c1fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:39.614097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e14a5-4449-485e-9c5c-5e6ebb43c1fc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:39.625310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 612/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:49:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:39.631023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c3e5a-c1e8-4d23-87b6-8fd411aa87dd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:39.688756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:39.754811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c3e5a-c1e8-4d23-87b6-8fd411aa87dd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:39.802335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda514ad-8511-48d6-9a46-2a63f05ed7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:39.803730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda514ad-8511-48d6-9a46-2a63f05ed7de None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:39.803730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda514ad-8511-48d6-9a46-2a63f05ed7de None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:39.803730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda514ad-8511-48d6-9a46-2a63f05ed7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:39.816836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda514ad-8511-48d6-9a46-2a63f05ed7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:39.818350 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda514ad-8511-48d6-9a46-2a63f05ed7de None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-dda514ad-8511-48d6-9a46-2a63f05ed7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:49:39.833922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:39.840927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:39.843022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:49:39.843022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:49:39.843022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:39.843022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:39.843022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 598/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:39.843405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e14a5-4449-485e-9c5c-5e6ebb43c1fc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:39.954314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:39.957656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e14a5-4449-485e-9c5c-5e6ebb43c1fc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:39.998292 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9334819-f6e5-429c-aab9-7af8ec505e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:39.998949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9334819-f6e5-429c-aab9-7af8ec505e67 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:39.999240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9334819-f6e5-429c-aab9-7af8ec505e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:39.999610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9334819-f6e5-429c-aab9-7af8ec505e67 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:40.000313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9334819-f6e5-429c-aab9-7af8ec505e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:40.000622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9334819-f6e5-429c-aab9-7af8ec505e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:40.001998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9334819-f6e5-429c-aab9-7af8ec505e67 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:40.016549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 613/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:49:40.022225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda514ad-8511-48d6-9a46-2a63f05ed7de None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:40.142277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda514ad-8511-48d6-9a46-2a63f05ed7de None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:40.143895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:40.182683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbaa95-938a-4016-8821-8b4eaadd30c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:40.183475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbaa95-938a-4016-8821-8b4eaadd30c3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:40.183869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbaa95-938a-4016-8821-8b4eaadd30c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:40.184205 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbaa95-938a-4016-8821-8b4eaadd30c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:40.193113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbaa95-938a-4016-8821-8b4eaadd30c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:40.194560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbaa95-938a-4016-8821-8b4eaadd30c3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-49bbaa95-938a-4016-8821-8b4eaadd30c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:49:40.207015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:40.212702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:40.213861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:40.213861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:40.213861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:49:40.213861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:49:40.213861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:49:40.213861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:49:40.213861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:49:40.213861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:49:40.213861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:49:40.213861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:40.213861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 599/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:49:40.214570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9334819-f6e5-429c-aab9-7af8ec505e67 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:40.332075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9334819-f6e5-429c-aab9-7af8ec505e67 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:40.370511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00c3ae-6df9-4e41-aca2-dda805972e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:40.371437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00c3ae-6df9-4e41-aca2-dda805972e84 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:40.371720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00c3ae-6df9-4e41-aca2-dda805972e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:40.371996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00c3ae-6df9-4e41-aca2-dda805972e84 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:40.373375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00c3ae-6df9-4e41-aca2-dda805972e84 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:40.373687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00c3ae-6df9-4e41-aca2-dda805972e84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:40.375143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00c3ae-6df9-4e41-aca2-dda805972e84 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:40.392840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 614/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jul 28 03:49:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:49:40.398686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbaa95-938a-4016-8821-8b4eaadd30c3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:40.517400 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbaa95-938a-4016-8821-8b4eaadd30c3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:40.563814 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478f6b0-2ebd-4c5b-8991-c1752cb7ff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:40.564901 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478f6b0-2ebd-4c5b-8991-c1752cb7ff3b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:40.565035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478f6b0-2ebd-4c5b-8991-c1752cb7ff3b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:40.565325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478f6b0-2ebd-4c5b-8991-c1752cb7ff3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:40.599382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:40.647493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478f6b0-2ebd-4c5b-8991-c1752cb7ff3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:40.648958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478f6b0-2ebd-4c5b-8991-c1752cb7ff3b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:40.764017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 600/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:49:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:49:40.770245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00c3ae-6df9-4e41-aca2-dda805972e84 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:40.791901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:40.894325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00c3ae-6df9-4e41-aca2-dda805972e84 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:40.941554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4e219-0904-4968-83b4-23be0b9d1016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:40.942203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4e219-0904-4968-83b4-23be0b9d1016 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:40.942491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4e219-0904-4968-83b4-23be0b9d1016 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:40.942770 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4e219-0904-4968-83b4-23be0b9d1016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:40.953149 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4e219-0904-4968-83b4-23be0b9d1016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:40.954601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4e219-0904-4968-83b4-23be0b9d1016 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-4ae4e219-0904-4968-83b4-23be0b9d1016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:49:40.973788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:40.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:40.976268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:49:40.976268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:49:40.976268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:40.976268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:40.976268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 615/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:40.981597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478f6b0-2ebd-4c5b-8991-c1752cb7ff3b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:41.063505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:41.112897 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478f6b0-2ebd-4c5b-8991-c1752cb7ff3b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:41.160905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472dc6b9-af46-4aef-a154-44df85c56352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:41.161574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc6b9-af46-4aef-a154-44df85c56352 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:41.161814 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc6b9-af46-4aef-a154-44df85c56352 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:41.162086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc6b9-af46-4aef-a154-44df85c56352 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:41.162773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dc6b9-af46-4aef-a154-44df85c56352 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:41.163118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dc6b9-af46-4aef-a154-44df85c56352 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:41.164496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dc6b9-af46-4aef-a154-44df85c56352 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:41.183675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 601/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:49:41.185093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4e219-0904-4968-83b4-23be0b9d1016 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:41.261725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:41.297193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4e219-0904-4968-83b4-23be0b9d1016 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:41.332115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9f957-f0f0-47a8-9167-a44a8f7abfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:41.332794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9f957-f0f0-47a8-9167-a44a8f7abfdc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:41.333181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9f957-f0f0-47a8-9167-a44a8f7abfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:41.333599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9f957-f0f0-47a8-9167-a44a8f7abfdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:41.340440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9f957-f0f0-47a8-9167-a44a8f7abfdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:41.342759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9f957-f0f0-47a8-9167-a44a8f7abfdc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-77f9f957-f0f0-47a8-9167-a44a8f7abfdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:49:41.353975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:41.355103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:41.356291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:49:41.356291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:49:41.356291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:49:41.356291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:49:41.356291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:49:41.356291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:49:41.356291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:49:41.356291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:41.362411 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 616/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:49:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:49:41.363640 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472dc6b9-af46-4aef-a154-44df85c56352 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:41.470633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472dc6b9-af46-4aef-a154-44df85c56352 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:41.495712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:41.509147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874beeb-51d1-4056-8bf1-e8185c72efee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:41.509935 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874beeb-51d1-4056-8bf1-e8185c72efee None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:41.510332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874beeb-51d1-4056-8bf1-e8185c72efee None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:41.510754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874beeb-51d1-4056-8bf1-e8185c72efee None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:41.511716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874beeb-51d1-4056-8bf1-e8185c72efee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:41.512061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874beeb-51d1-4056-8bf1-e8185c72efee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:41.513996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874beeb-51d1-4056-8bf1-e8185c72efee None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:41.529997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 602/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:49:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:41.536977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9f957-f0f0-47a8-9167-a44a8f7abfdc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:41.656896 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9f957-f0f0-47a8-9167-a44a8f7abfdc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:41.687153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:41.694416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5596e-4724-4ed7-9dd9-414abf4c589f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:41.695094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5596e-4724-4ed7-9dd9-414abf4c589f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:41.695402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5596e-4724-4ed7-9dd9-414abf4c589f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:41.695645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5596e-4724-4ed7-9dd9-414abf4c589f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:41.696438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5596e-4724-4ed7-9dd9-414abf4c589f None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=551281a31ce24e2f9b81b36f69419311) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:41.696739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5596e-4724-4ed7-9dd9-414abf4c589f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:41.698195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5596e-4724-4ed7-9dd9-414abf4c589f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:41.712549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 617/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:49:41 2022] GET /identity/v3/role_assignments?user.id=551281a31ce24e2f9b81b36f69419311&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:43.686400 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15984e92-34f7-43ad-ab34-2d336d7ecd03 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:43.687007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15984e92-34f7-43ad-ab34-2d336d7ecd03 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:43.688975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15984e92-34f7-43ad-ab34-2d336d7ecd03 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:43.690359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 603/1220] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:49:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:49:43.696105 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a84b1f-5efc-455c-bd56-df94801de059 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:43.696560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a84b1f-5efc-455c-bd56-df94801de059 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:43.696799 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a84b1f-5efc-455c-bd56-df94801de059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:43.714633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-78a84b1f-5efc-455c-bd56-df94801de059 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:49:43.715861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-78a84b1f-5efc-455c-bd56-df94801de059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:49:43.788414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 618/1221] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:49:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:49:43.809788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15984e92-34f7-43ad-ab34-2d336d7ecd03 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:43.922353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15984e92-34f7-43ad-ab34-2d336d7ecd03 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:43.960997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27432ee-356e-48a0-a494-c09a246fc305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:43.961646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27432ee-356e-48a0-a494-c09a246fc305 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:43.961945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27432ee-356e-48a0-a494-c09a246fc305 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:43.962208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27432ee-356e-48a0-a494-c09a246fc305 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:43.973095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27432ee-356e-48a0-a494-c09a246fc305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d27432ee-356e-48a0-a494-c09a246fc305 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:49:43.977921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:43.979021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:43.979868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 604/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:49:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:43.985462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a84b1f-5efc-455c-bd56-df94801de059 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:44.091109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a84b1f-5efc-455c-bd56-df94801de059 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:44.114354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:44.125948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e61ae-277c-4ace-abf2-f85e0b7e01ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:44.126568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e61ae-277c-4ace-abf2-f85e0b7e01ce None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:44.126874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e61ae-277c-4ace-abf2-f85e0b7e01ce None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:44.127147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e61ae-277c-4ace-abf2-f85e0b7e01ce None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:44.127858 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e61ae-277c-4ace-abf2-f85e0b7e01ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:44.128165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e61ae-277c-4ace-abf2-f85e0b7e01ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:44.129519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e61ae-277c-4ace-abf2-f85e0b7e01ce None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:44.137731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 619/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:49:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:44.143513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27432ee-356e-48a0-a494-c09a246fc305 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:44.194720 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:44.259981 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27432ee-356e-48a0-a494-c09a246fc305 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:44.296807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcf19b-5368-4866-9737-447b4d7b16f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:44.297447 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcf19b-5368-4866-9737-447b4d7b16f3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:44.297726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcf19b-5368-4866-9737-447b4d7b16f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:44.298016 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcf19b-5368-4866-9737-447b4d7b16f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:44.308013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcf19b-5368-4866-9737-447b4d7b16f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:44.309431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcf19b-5368-4866-9737-447b4d7b16f3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-bddcf19b-5368-4866-9737-447b4d7b16f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:49:44.323719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:44.324899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:44.326046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:49:44.326046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:49:44.326046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:44.326046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:44.326046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 605/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:49:44.332306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e61ae-277c-4ace-abf2-f85e0b7e01ce None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:44.442406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e61ae-277c-4ace-abf2-f85e0b7e01ce None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:44.482076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8180d1-887f-4838-9e7e-c54ed49ff8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:44.483255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8180d1-887f-4838-9e7e-c54ed49ff8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:44.483545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8180d1-887f-4838-9e7e-c54ed49ff8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:44.483824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8180d1-887f-4838-9e7e-c54ed49ff8e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:44.484522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8180d1-887f-4838-9e7e-c54ed49ff8e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:44.484828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8180d1-887f-4838-9e7e-c54ed49ff8e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:44.486196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8180d1-887f-4838-9e7e-c54ed49ff8e5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:44.498908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 620/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:44.504483 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcf19b-5368-4866-9737-447b4d7b16f3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:44.609052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:44.616093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcf19b-5368-4866-9737-447b4d7b16f3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:44.654030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455f39e-07c5-4201-b2c3-dabd66798a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:44.654436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455f39e-07c5-4201-b2c3-dabd66798a58 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:44.654696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455f39e-07c5-4201-b2c3-dabd66798a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:44.655004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455f39e-07c5-4201-b2c3-dabd66798a58 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:44.663306 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e455f39e-07c5-4201-b2c3-dabd66798a58 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:44.664759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e455f39e-07c5-4201-b2c3-dabd66798a58 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-e455f39e-07c5-4201-b2c3-dabd66798a58 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:49:44.676709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:44.677863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:44.679036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:44.679036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:44.679036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:49:44.679036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:49:44.679036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:49:44.679036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:49:44.679036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:49:44.679036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:49:44.679036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:49:44.679036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:44.679036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 606/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:49:44.683864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8180d1-887f-4838-9e7e-c54ed49ff8e5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:44.737705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:44.797733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8180d1-887f-4838-9e7e-c54ed49ff8e5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:44.836768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4f78b-058e-425d-aa01-5a5a61f069a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:44.837386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4f78b-058e-425d-aa01-5a5a61f069a7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:44.837670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4f78b-058e-425d-aa01-5a5a61f069a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:44.837944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4f78b-058e-425d-aa01-5a5a61f069a7 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:44.838738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4f78b-058e-425d-aa01-5a5a61f069a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:44.839094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4f78b-058e-425d-aa01-5a5a61f069a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:44.840495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4f78b-058e-425d-aa01-5a5a61f069a7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:44.860161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 621/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jul 28 03:49:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:49:44.866071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455f39e-07c5-4201-b2c3-dabd66798a58 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:44.977034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455f39e-07c5-4201-b2c3-dabd66798a58 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:45.012203 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8e162-58cd-40db-849b-264e6e723f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:45.012774 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8e162-58cd-40db-849b-264e6e723f79 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:45.013057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8e162-58cd-40db-849b-264e6e723f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:45.013878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8e162-58cd-40db-849b-264e6e723f79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:45.025179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8e162-58cd-40db-849b-264e6e723f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:45.026746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8e162-58cd-40db-849b-264e6e723f79 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a0f8e162-58cd-40db-849b-264e6e723f79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:49:45.042204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:45.043477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:45.044843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:49:45.044843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:49:45.044843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:45.044843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:45.044843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 607/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:45.049412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4f78b-058e-425d-aa01-5a5a61f069a7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:45.156106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4f78b-058e-425d-aa01-5a5a61f069a7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:45.158695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:45.194354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299608b3-12d4-4753-84c2-342523d6375f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:45.195107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299608b3-12d4-4753-84c2-342523d6375f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:45.195389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299608b3-12d4-4753-84c2-342523d6375f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:45.195662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299608b3-12d4-4753-84c2-342523d6375f None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:45.196356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299608b3-12d4-4753-84c2-342523d6375f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:45.196660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299608b3-12d4-4753-84c2-342523d6375f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:45.198028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299608b3-12d4-4753-84c2-342523d6375f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:45.210323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 622/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:45.216028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8e162-58cd-40db-849b-264e6e723f79 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:45.226740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:45.329717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8e162-58cd-40db-849b-264e6e723f79 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:45.365094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2ae9f-2d3a-440b-9783-c93c33419d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:45.365743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2ae9f-2d3a-440b-9783-c93c33419d06 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:45.366018 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2ae9f-2d3a-440b-9783-c93c33419d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:45.366293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2ae9f-2d3a-440b-9783-c93c33419d06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:45.373751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2ae9f-2d3a-440b-9783-c93c33419d06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:45.375239 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2ae9f-2d3a-440b-9783-c93c33419d06 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-94b2ae9f-2d3a-440b-9783-c93c33419d06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:49:45.386512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:45.387547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:45.389223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:49:45.389223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:49:45.389223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:49:45.389223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:49:45.389223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:49:45.389223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:49:45.389223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:49:45.389223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:45.389223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 608/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:49:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:45.394635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299608b3-12d4-4753-84c2-342523d6375f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:45.505125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299608b3-12d4-4753-84c2-342523d6375f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:45.543149 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc580e4-19cb-4f70-9b06-055532872b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:45.543775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc580e4-19cb-4f70-9b06-055532872b7b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:45.544059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc580e4-19cb-4f70-9b06-055532872b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:45.544359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc580e4-19cb-4f70-9b06-055532872b7b None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:45.545557 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc580e4-19cb-4f70-9b06-055532872b7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:45.545868 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc580e4-19cb-4f70-9b06-055532872b7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:45.547367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc580e4-19cb-4f70-9b06-055532872b7b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:45.565276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 623/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:49:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:45.566486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2ae9f-2d3a-440b-9783-c93c33419d06 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:45.643076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:45.684556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2ae9f-2d3a-440b-9783-c93c33419d06 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:45.721648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15851e2-7714-44dc-8e78-6a273638e6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:45.722322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15851e2-7714-44dc-8e78-6a273638e6ef None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:45.722604 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15851e2-7714-44dc-8e78-6a273638e6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:45.722909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15851e2-7714-44dc-8e78-6a273638e6ef None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/551281a31ce24e2f9b81b36f69419311/roles/f5a9eb03e88a4e66bb9a8b6180009f72` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:45.740387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15851e2-7714-44dc-8e78-6a273638e6ef None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=551281a31ce24e2f9b81b36f69419311, role_id=f5a9eb03e88a4e66bb9a8b6180009f72)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:45.742352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15851e2-7714-44dc-8e78-6a273638e6ef None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:45.756835 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 609/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:49:45 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/551281a31ce24e2f9b81b36f69419311/roles/f5a9eb03e88a4e66bb9a8b6180009f72 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:49:47.909071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efb9e5-7c26-4c00-acec-18b3bdaa0fe6 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:47.909628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efb9e5-7c26-4c00-acec-18b3bdaa0fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:47.909628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efb9e5-7c26-4c00-acec-18b3bdaa0fe6 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:47.911315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 624/1233] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:49:47.916468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814531e-1897-4e06-be1f-0239bf89c823 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:47.916885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814531e-1897-4e06-be1f-0239bf89c823 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:47.917055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814531e-1897-4e06-be1f-0239bf89c823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:47.937473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a814531e-1897-4e06-be1f-0239bf89c823 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:49:47.938685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a814531e-1897-4e06-be1f-0239bf89c823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:49:48.007895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 610/1234] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:49:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:49:48.031386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efb9e5-7c26-4c00-acec-18b3bdaa0fe6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:48.058181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:48.140500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efb9e5-7c26-4c00-acec-18b3bdaa0fe6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:48.178626 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f633853-c80d-49a4-8d30-83b06d700190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:48.179283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633853-c80d-49a4-8d30-83b06d700190 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:48.179569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633853-c80d-49a4-8d30-83b06d700190 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:48.179875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633853-c80d-49a4-8d30-83b06d700190 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:48.191486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633853-c80d-49a4-8d30-83b06d700190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-7f633853-c80d-49a4-8d30-83b06d700190 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:49:48.196385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:48.202876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:48.203957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 625/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jul 28 03:49:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:48.204655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814531e-1897-4e06-be1f-0239bf89c823 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:48.260042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:48.323208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814531e-1897-4e06-be1f-0239bf89c823 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:48.361108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e27548-6842-45a1-89a0-b462ffee6abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:48.361718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e27548-6842-45a1-89a0-b462ffee6abb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:48.361998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e27548-6842-45a1-89a0-b462ffee6abb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:48.362268 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e27548-6842-45a1-89a0-b462ffee6abb None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:48.363010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e27548-6842-45a1-89a0-b462ffee6abb None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:48.363421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e27548-6842-45a1-89a0-b462ffee6abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:48.364659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e27548-6842-45a1-89a0-b462ffee6abb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:48.372888 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 611/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jul 28 03:49:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:48.378529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f633853-c80d-49a4-8d30-83b06d700190 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:48.469404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:48.486703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f633853-c80d-49a4-8d30-83b06d700190 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:48.521861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee467f-b286-4260-bcd1-8fa1c5bb2de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:48.522536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee467f-b286-4260-bcd1-8fa1c5bb2de4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:48.522883 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee467f-b286-4260-bcd1-8fa1c5bb2de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:48.523231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee467f-b286-4260-bcd1-8fa1c5bb2de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:48.534186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee467f-b286-4260-bcd1-8fa1c5bb2de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:48.535968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee467f-b286-4260-bcd1-8fa1c5bb2de4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-afee467f-b286-4260-bcd1-8fa1c5bb2de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:49:48.553211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:48.554188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:48.555125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:49:48.555125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:49:48.555125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:48.555125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:48.555125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 626/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:48.561686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e27548-6842-45a1-89a0-b462ffee6abb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:48.675394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e27548-6842-45a1-89a0-b462ffee6abb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:48.684515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:48.712311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd9adb-4a72-402d-99f0-e69190ce8543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:48.712937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd9adb-4a72-402d-99f0-e69190ce8543 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:48.713228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd9adb-4a72-402d-99f0-e69190ce8543 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:48.713501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd9adb-4a72-402d-99f0-e69190ce8543 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:48.714193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd9adb-4a72-402d-99f0-e69190ce8543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:48.714499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd9adb-4a72-402d-99f0-e69190ce8543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:48.715953 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd9adb-4a72-402d-99f0-e69190ce8543 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:48.734963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 612/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:49:48.736134 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee467f-b286-4260-bcd1-8fa1c5bb2de4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:48.855339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee467f-b286-4260-bcd1-8fa1c5bb2de4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:48.893540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f264a5-6b5b-4294-9a3c-62f3cb7ad4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:48.894210 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f264a5-6b5b-4294-9a3c-62f3cb7ad4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:48.894493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f264a5-6b5b-4294-9a3c-62f3cb7ad4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:48.894763 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f264a5-6b5b-4294-9a3c-62f3cb7ad4e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:48.897906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:48.905160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f264a5-6b5b-4294-9a3c-62f3cb7ad4e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:48.907148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f264a5-6b5b-4294-9a3c-62f3cb7ad4e3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-05f264a5-6b5b-4294-9a3c-62f3cb7ad4e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:49:48.920580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:48.922183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:48.923367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:48.923367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:48.923367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:49:48.923367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:49:48.923367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:49:48.923367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:49:48.923367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:49:48.923367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:49:48.923367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:49:48.923367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:48.923367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 627/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:48.927589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd9adb-4a72-402d-99f0-e69190ce8543 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:49.038166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd9adb-4a72-402d-99f0-e69190ce8543 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:49.079130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561fbd2-fc13-4780-9498-365c72837ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:49.080449 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561fbd2-fc13-4780-9498-365c72837ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:49.080762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561fbd2-fc13-4780-9498-365c72837ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:49.081024 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561fbd2-fc13-4780-9498-365c72837ea4 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:49.081824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561fbd2-fc13-4780-9498-365c72837ea4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:49.082149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561fbd2-fc13-4780-9498-365c72837ea4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:49.083643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561fbd2-fc13-4780-9498-365c72837ea4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:49.101634 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 613/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jul 28 03:49:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:49.108082 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f264a5-6b5b-4294-9a3c-62f3cb7ad4e3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:49.222172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f264a5-6b5b-4294-9a3c-62f3cb7ad4e3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:49.259226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3023b-7d9d-495d-a3e5-d7fe3480b7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:49.259879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3023b-7d9d-495d-a3e5-d7fe3480b7cb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:49.260163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3023b-7d9d-495d-a3e5-d7fe3480b7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:49.260436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3023b-7d9d-495d-a3e5-d7fe3480b7cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:49.335071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:49.343196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3023b-7d9d-495d-a3e5-d7fe3480b7cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:49.344672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3023b-7d9d-495d-a3e5-d7fe3480b7cb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:49.463103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 628/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:49:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:49:49.470722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561fbd2-fc13-4780-9498-365c72837ea4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:49.509432 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:49.594152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561fbd2-fc13-4780-9498-365c72837ea4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:49.631876 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540cfc7-be5f-4321-894d-a497e2953048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:49.632615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540cfc7-be5f-4321-894d-a497e2953048 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:49.632902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540cfc7-be5f-4321-894d-a497e2953048 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:49.633179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540cfc7-be5f-4321-894d-a497e2953048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:49.644740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540cfc7-be5f-4321-894d-a497e2953048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:49.646199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540cfc7-be5f-4321-894d-a497e2953048 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8540cfc7-be5f-4321-894d-a497e2953048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:49:49.662952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:49.664009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:49.665096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:49:49.665096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:49:49.665096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:49.665096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:49.665096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 614/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:49.671036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3023b-7d9d-495d-a3e5-d7fe3480b7cb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:49.771198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:49.777090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3023b-7d9d-495d-a3e5-d7fe3480b7cb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:49.812074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da93f5-4497-4be5-a37d-e207c1d740f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:49.812698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da93f5-4497-4be5-a37d-e207c1d740f5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:49.812984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da93f5-4497-4be5-a37d-e207c1d740f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:49.813263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da93f5-4497-4be5-a37d-e207c1d740f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:49.813959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da93f5-4497-4be5-a37d-e207c1d740f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:49.814269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da93f5-4497-4be5-a37d-e207c1d740f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:49.815662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da93f5-4497-4be5-a37d-e207c1d740f5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:49.828175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 629/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:49.834532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540cfc7-be5f-4321-894d-a497e2953048 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:49.921247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:49.944832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540cfc7-be5f-4321-894d-a497e2953048 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:49.988855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959c388-5f11-42bc-9671-7dea619c19ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:49.989495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959c388-5f11-42bc-9671-7dea619c19ad None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:49.989776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959c388-5f11-42bc-9671-7dea619c19ad None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:49.990049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959c388-5f11-42bc-9671-7dea619c19ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:49.997181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959c388-5f11-42bc-9671-7dea619c19ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:49.998665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959c388-5f11-42bc-9671-7dea619c19ad None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5959c388-5f11-42bc-9671-7dea619c19ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:49:50.009853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:50.010866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:50.011764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:49:50.011764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:49:50.011764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:49:50.011764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:49:50.011764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:49:50.011764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:49:50.011764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:49:50.011764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:50.011764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 615/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:50.017364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da93f5-4497-4be5-a37d-e207c1d740f5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:50.132035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da93f5-4497-4be5-a37d-e207c1d740f5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:50.169890 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555592c-2f09-4775-8e02-53f46eb0a766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:50.170641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555592c-2f09-4775-8e02-53f46eb0a766 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:50.171011 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555592c-2f09-4775-8e02-53f46eb0a766 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:50.171356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555592c-2f09-4775-8e02-53f46eb0a766 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:50.172159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555592c-2f09-4775-8e02-53f46eb0a766 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:50.172531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555592c-2f09-4775-8e02-53f46eb0a766 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:50.174207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555592c-2f09-4775-8e02-53f46eb0a766 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:50.188669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 630/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:49:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:49:50.195581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959c388-5f11-42bc-9671-7dea619c19ad None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:50.314245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959c388-5f11-42bc-9671-7dea619c19ad None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:50.351463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1fe59-6cdb-464c-8ff7-fe16f1471bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:50.352193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1fe59-6cdb-464c-8ff7-fe16f1471bcc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:50.352468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1fe59-6cdb-464c-8ff7-fe16f1471bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:50.352825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1fe59-6cdb-464c-8ff7-fe16f1471bcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:50.353686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1fe59-6cdb-464c-8ff7-fe16f1471bcc None admin] RBAC: Adding query filter params (role.id=f5a9eb03e88a4e66bb9a8b6180009f72, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=551281a31ce24e2f9b81b36f69419311) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:50.354091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1fe59-6cdb-464c-8ff7-fe16f1471bcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:50.355842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1fe59-6cdb-464c-8ff7-fe16f1471bcc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:50.363455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 616/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:49:50 2022] GET /identity/v3/role_assignments?user.id=551281a31ce24e2f9b81b36f69419311&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=f5a9eb03e88a4e66bb9a8b6180009f72 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:52.431106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc68ad0-b37c-4bb1-b0be-42ca01be8542 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:52.431648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc68ad0-b37c-4bb1-b0be-42ca01be8542 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:52.431648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc68ad0-b37c-4bb1-b0be-42ca01be8542 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:52.433225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 631/1247] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:49:52.439140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d297a2c-7ea8-4330-ab22-9ac5ba2db800 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:52.439591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d297a2c-7ea8-4330-ab22-9ac5ba2db800 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:52.439692 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d297a2c-7ea8-4330-ab22-9ac5ba2db800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:52.446477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:52.460870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4d297a2c-7ea8-4330-ab22-9ac5ba2db800 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:49:52.462085 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4d297a2c-7ea8-4330-ab22-9ac5ba2db800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:49:52.536605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 617/1248] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:49:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:49:52.564254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc68ad0-b37c-4bb1-b0be-42ca01be8542 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:52.585666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:52.675554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc68ad0-b37c-4bb1-b0be-42ca01be8542 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:52.714724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93727ac-4fe8-4354-9f1a-fe93d9e6300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:52.715327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93727ac-4fe8-4354-9f1a-fe93d9e6300f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:52.715613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93727ac-4fe8-4354-9f1a-fe93d9e6300f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:52.716413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93727ac-4fe8-4354-9f1a-fe93d9e6300f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:52.727616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93727ac-4fe8-4354-9f1a-fe93d9e6300f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d93727ac-4fe8-4354-9f1a-fe93d9e6300f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:49:52.733116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:52.734242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:49:52.735340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:52.736088 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 632/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:52.742972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d297a2c-7ea8-4330-ab22-9ac5ba2db800 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:52.859607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d297a2c-7ea8-4330-ab22-9ac5ba2db800 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:52.896177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:52.904138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084c659-4ce1-4dc6-829d-5c2300938df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:52.904852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084c659-4ce1-4dc6-829d-5c2300938df3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:52.905210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084c659-4ce1-4dc6-829d-5c2300938df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:52.905568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084c659-4ce1-4dc6-829d-5c2300938df3 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:52.906624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084c659-4ce1-4dc6-829d-5c2300938df3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:52.907077 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084c659-4ce1-4dc6-829d-5c2300938df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:52.909012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084c659-4ce1-4dc6-829d-5c2300938df3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:52.918960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 618/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:49:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:52.925676 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93727ac-4fe8-4354-9f1a-fe93d9e6300f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:53.017667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:53.044977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93727ac-4fe8-4354-9f1a-fe93d9e6300f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:53.086730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5c41b-bf5c-4f06-9610-acd232b0b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:53.087520 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5c41b-bf5c-4f06-9610-acd232b0b299 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:53.087808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5c41b-bf5c-4f06-9610-acd232b0b299 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:53.088091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5c41b-bf5c-4f06-9610-acd232b0b299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:53.101808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5c41b-bf5c-4f06-9610-acd232b0b299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:53.103453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5c41b-bf5c-4f06-9610-acd232b0b299 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d4d5c41b-bf5c-4f06-9610-acd232b0b299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:49:53.120964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:53.122933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:53.123938 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:49:53.123938 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:49:53.123938 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:53.123938 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:53.123938 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 633/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:53.130146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084c659-4ce1-4dc6-829d-5c2300938df3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:53.250103 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084c659-4ce1-4dc6-829d-5c2300938df3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:53.290835 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357ea03-b7cd-4cb9-ad0e-a438a1eb35e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:53.292562 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357ea03-b7cd-4cb9-ad0e-a438a1eb35e9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:53.292852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357ea03-b7cd-4cb9-ad0e-a438a1eb35e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:53.293161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357ea03-b7cd-4cb9-ad0e-a438a1eb35e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:53.293861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357ea03-b7cd-4cb9-ad0e-a438a1eb35e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:53.294376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357ea03-b7cd-4cb9-ad0e-a438a1eb35e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:53.296234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357ea03-b7cd-4cb9-ad0e-a438a1eb35e9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:53.312454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 619/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:53.319169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5c41b-bf5c-4f06-9610-acd232b0b299 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:53.436696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5c41b-bf5c-4f06-9610-acd232b0b299 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:53.468728 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:53.477965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c829a-1590-4939-94ab-ecb51a42c4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:53.478603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c829a-1590-4939-94ab-ecb51a42c4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:53.478913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c829a-1590-4939-94ab-ecb51a42c4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:53.479198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c829a-1590-4939-94ab-ecb51a42c4d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:53.487275 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c829a-1590-4939-94ab-ecb51a42c4d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:53.488699 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c829a-1590-4939-94ab-ecb51a42c4d0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-410c829a-1590-4939-94ab-ecb51a42c4d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:49:53.501001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:53.502037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:53.503036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:53.503036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:53.503036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:49:53.503036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:49:53.503036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:49:53.503036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:49:53.503036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:49:53.503036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:49:53.503036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:49:53.503036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:53.507801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 634/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:53.508959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357ea03-b7cd-4cb9-ad0e-a438a1eb35e9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:53.575400 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:53.629249 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357ea03-b7cd-4cb9-ad0e-a438a1eb35e9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:53.665250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ebdff-c3ac-49d4-ab5c-37b73e57ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:53.665857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ebdff-c3ac-49d4-ab5c-37b73e57ade2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:53.666136 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ebdff-c3ac-49d4-ab5c-37b73e57ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:53.666407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ebdff-c3ac-49d4-ab5c-37b73e57ade2 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:53.667221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ebdff-c3ac-49d4-ab5c-37b73e57ade2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:53.667531 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ebdff-c3ac-49d4-ab5c-37b73e57ade2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:53.669550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ebdff-c3ac-49d4-ab5c-37b73e57ade2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:53.687137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 620/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jul 28 03:49:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:49:53.693185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c829a-1590-4939-94ab-ecb51a42c4d0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:53.806505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c829a-1590-4939-94ab-ecb51a42c4d0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:53.848487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bb20e-6835-44d3-859a-14bd88c448c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:53.849337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bb20e-6835-44d3-859a-14bd88c448c7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:53.849666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bb20e-6835-44d3-859a-14bd88c448c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:53.850360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bb20e-6835-44d3-859a-14bd88c448c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:53.914882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:53.928400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bb20e-6835-44d3-859a-14bd88c448c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:53.929854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bb20e-6835-44d3-859a-14bd88c448c7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:54.043478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 635/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:49:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:49:54.050584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ebdff-c3ac-49d4-ab5c-37b73e57ade2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:54.161525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ebdff-c3ac-49d4-ab5c-37b73e57ade2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:54.175080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:54.198475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c264b-a7c3-41b7-8e5a-e2a52f0cd922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:54.199180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c264b-a7c3-41b7-8e5a-e2a52f0cd922 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:54.199460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c264b-a7c3-41b7-8e5a-e2a52f0cd922 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:54.199771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c264b-a7c3-41b7-8e5a-e2a52f0cd922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:54.211161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c264b-a7c3-41b7-8e5a-e2a52f0cd922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:54.212716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c264b-a7c3-41b7-8e5a-e2a52f0cd922 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b93c264b-a7c3-41b7-8e5a-e2a52f0cd922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:49:54.228254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:54.229324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:54.230243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:49:54.230243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:49:54.230243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:54.230243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:54.230243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 621/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:54.237759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bb20e-6835-44d3-859a-14bd88c448c7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:54.343326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:54.358580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bb20e-6835-44d3-859a-14bd88c448c7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:54.395185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92dcffe-b710-4f21-8b51-7d42703205b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:54.395805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dcffe-b710-4f21-8b51-7d42703205b7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:54.396085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dcffe-b710-4f21-8b51-7d42703205b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:54.396355 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dcffe-b710-4f21-8b51-7d42703205b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:54.397050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92dcffe-b710-4f21-8b51-7d42703205b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:54.397368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92dcffe-b710-4f21-8b51-7d42703205b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:54.398837 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92dcffe-b710-4f21-8b51-7d42703205b7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:54.411522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 636/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:54.417456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c264b-a7c3-41b7-8e5a-e2a52f0cd922 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:54.519577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c264b-a7c3-41b7-8e5a-e2a52f0cd922 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:54.560062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd1df2-80d8-4e89-8236-63de21df95b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:54.560696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd1df2-80d8-4e89-8236-63de21df95b4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:54.560977 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd1df2-80d8-4e89-8236-63de21df95b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:54.561247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd1df2-80d8-4e89-8236-63de21df95b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:54.568007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd1df2-80d8-4e89-8236-63de21df95b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:54.569460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd1df2-80d8-4e89-8236-63de21df95b4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2ddd1df2-80d8-4e89-8236-63de21df95b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:49:54.585559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:54.586755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:54.588200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:49:54.588200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:49:54.588200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:49:54.588200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:49:54.588200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:49:54.588200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:49:54.588200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:49:54.588200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:49:54.588200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 622/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:49:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 03:49:54.592664 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92dcffe-b710-4f21-8b51-7d42703205b7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:54.699852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92dcffe-b710-4f21-8b51-7d42703205b7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:54.735611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa5021-334a-4288-bfc2-0990987dd25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:54.736294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa5021-334a-4288-bfc2-0990987dd25c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:54.736530 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa5021-334a-4288-bfc2-0990987dd25c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:54.736805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa5021-334a-4288-bfc2-0990987dd25c None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:54.737552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa5021-334a-4288-bfc2-0990987dd25c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:54.737865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa5021-334a-4288-bfc2-0990987dd25c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:54.739343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa5021-334a-4288-bfc2-0990987dd25c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:54.754811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 637/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:49:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:54.758443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd1df2-80d8-4e89-8236-63de21df95b4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:54.785444 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:54.866877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd1df2-80d8-4e89-8236-63de21df95b4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:54.902726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9604b2-ec61-43cc-9c8a-b627ebf63a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:54.903894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9604b2-ec61-43cc-9c8a-b627ebf63a60 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:54.904196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9604b2-ec61-43cc-9c8a-b627ebf63a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:54.904446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9604b2-ec61-43cc-9c8a-b627ebf63a60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:54.905228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9604b2-ec61-43cc-9c8a-b627ebf63a60 None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=551281a31ce24e2f9b81b36f69419311) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:54.905540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9604b2-ec61-43cc-9c8a-b627ebf63a60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:54.907008 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9604b2-ec61-43cc-9c8a-b627ebf63a60 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:54.912940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 623/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:49:54 2022] GET /identity/v3/role_assignments?user.id=551281a31ce24e2f9b81b36f69419311&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=a37ae24c516b48c69e13721c013134a2 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:56.905340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bcbaf-ccdc-4ea0-be60-59b83f8c945a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:56.905910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bcbaf-ccdc-4ea0-be60-59b83f8c945a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:56.905910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bcbaf-ccdc-4ea0-be60-59b83f8c945a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:56.907317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 638/1261] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:49:56.912709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54630e0f-8c8f-42f3-9e73-8e1bdc45a56d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:56.913121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54630e0f-8c8f-42f3-9e73-8e1bdc45a56d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:56.913263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54630e0f-8c8f-42f3-9e73-8e1bdc45a56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:56.931807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-54630e0f-8c8f-42f3-9e73-8e1bdc45a56d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:49:56.933046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-54630e0f-8c8f-42f3-9e73-8e1bdc45a56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:49:56.998106 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 624/1262] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:49:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:49:57.020265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bcbaf-ccdc-4ea0-be60-59b83f8c945a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:57.031117 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:57.119881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bcbaf-ccdc-4ea0-be60-59b83f8c945a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:57.152623 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dad25-1eb7-49b1-9fc4-2f92706702de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:57.153835 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dad25-1eb7-49b1-9fc4-2f92706702de None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:57.154144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dad25-1eb7-49b1-9fc4-2f92706702de None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:57.154512 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dad25-1eb7-49b1-9fc4-2f92706702de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:57.164674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dad25-1eb7-49b1-9fc4-2f92706702de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-179dad25-1eb7-49b1-9fc4-2f92706702de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:49:57.169734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:57.170933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:57.172066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 639/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:57.176995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54630e0f-8c8f-42f3-9e73-8e1bdc45a56d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:57.195597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:57.284036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54630e0f-8c8f-42f3-9e73-8e1bdc45a56d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:57.322665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e4b52-ca83-48d8-b58d-45f5bed63cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:57.323299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e4b52-ca83-48d8-b58d-45f5bed63cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:57.323579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e4b52-ca83-48d8-b58d-45f5bed63cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:57.323850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e4b52-ca83-48d8-b58d-45f5bed63cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:57.324553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e4b52-ca83-48d8-b58d-45f5bed63cf5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:57.324856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e4b52-ca83-48d8-b58d-45f5bed63cf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:57.326618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e4b52-ca83-48d8-b58d-45f5bed63cf5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:57.334490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 625/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:49:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:57.340232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dad25-1eb7-49b1-9fc4-2f92706702de None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:57.406006 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:57.439201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dad25-1eb7-49b1-9fc4-2f92706702de None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:57.471630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e89647-fc0a-4d82-8af0-b58f3b53df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:57.472254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e89647-fc0a-4d82-8af0-b58f3b53df61 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:57.472537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e89647-fc0a-4d82-8af0-b58f3b53df61 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:57.472810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e89647-fc0a-4d82-8af0-b58f3b53df61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:57.483165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e89647-fc0a-4d82-8af0-b58f3b53df61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:57.484598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e89647-fc0a-4d82-8af0-b58f3b53df61 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-51e89647-fc0a-4d82-8af0-b58f3b53df61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:49:57.499376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:57.500424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:57.501730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:49:57.501730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:49:57.501730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:57.501730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:57.501730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 640/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:57.506193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e4b52-ca83-48d8-b58d-45f5bed63cf5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:57.584198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:57.602763 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e4b52-ca83-48d8-b58d-45f5bed63cf5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:57.635283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7cfb3-09f2-4e31-bb09-c0b5e18956b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:57.635884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7cfb3-09f2-4e31-bb09-c0b5e18956b6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:57.636161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7cfb3-09f2-4e31-bb09-c0b5e18956b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:57.636436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7cfb3-09f2-4e31-bb09-c0b5e18956b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:57.637111 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7cfb3-09f2-4e31-bb09-c0b5e18956b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:57.637412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7cfb3-09f2-4e31-bb09-c0b5e18956b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:57.638993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7cfb3-09f2-4e31-bb09-c0b5e18956b6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:57.650041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 626/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:57.656300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e89647-fc0a-4d82-8af0-b58f3b53df61 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:57.754472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e89647-fc0a-4d82-8af0-b58f3b53df61 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:57.773111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:57.787353 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a9675-ecdb-4045-894e-d166d9599b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:57.787974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a9675-ecdb-4045-894e-d166d9599b0c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:57.788253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a9675-ecdb-4045-894e-d166d9599b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:57.788524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a9675-ecdb-4045-894e-d166d9599b0c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:57.796279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a9675-ecdb-4045-894e-d166d9599b0c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:57.797712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a9675-ecdb-4045-894e-d166d9599b0c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e41a9675-ecdb-4045-894e-d166d9599b0c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:49:57.808426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:57.809650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:57.810633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:57.810633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:57.810633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:49:57.810633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:49:57.810633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:49:57.810633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:49:57.810633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:49:57.810633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:49:57.810633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:49:57.810633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:57.810633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 641/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:57.815301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7cfb3-09f2-4e31-bb09-c0b5e18956b6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:57.917562 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7cfb3-09f2-4e31-bb09-c0b5e18956b6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:57.950269 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4e7d4-3fda-4e42-a6a0-e2abaca63d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:57.950894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4e7d4-3fda-4e42-a6a0-e2abaca63d6b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:57.951174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4e7d4-3fda-4e42-a6a0-e2abaca63d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:57.951441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4e7d4-3fda-4e42-a6a0-e2abaca63d6b None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:57.952216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4e7d4-3fda-4e42-a6a0-e2abaca63d6b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:57.952516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4e7d4-3fda-4e42-a6a0-e2abaca63d6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:57.953923 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4e7d4-3fda-4e42-a6a0-e2abaca63d6b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:57.962156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:57.971975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 627/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jul 28 03:49:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:57.977495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a9675-ecdb-4045-894e-d166d9599b0c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:58.074158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a9675-ecdb-4045-894e-d166d9599b0c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:58.106641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d80f9b-5a3e-461c-a45e-af3d94ba3bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:58.107282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d80f9b-5a3e-461c-a45e-af3d94ba3bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:58.107563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d80f9b-5a3e-461c-a45e-af3d94ba3bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:58.107835 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d80f9b-5a3e-461c-a45e-af3d94ba3bf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:58.117803 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d80f9b-5a3e-461c-a45e-af3d94ba3bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:58.119254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d80f9b-5a3e-461c-a45e-af3d94ba3bf4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-57d80f9b-5a3e-461c-a45e-af3d94ba3bf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:49:58.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:58.133741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:58.134955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:49:58.134955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:49:58.134955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:49:58.134955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:58.134955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 642/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:58.140052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4e7d4-3fda-4e42-a6a0-e2abaca63d6b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:58.235285 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4e7d4-3fda-4e42-a6a0-e2abaca63d6b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:58.271866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56782b-b355-4375-b792-7182ea08bf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:58.272478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56782b-b355-4375-b792-7182ea08bf64 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:58.272768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56782b-b355-4375-b792-7182ea08bf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:58.273040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56782b-b355-4375-b792-7182ea08bf64 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:58.273724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56782b-b355-4375-b792-7182ea08bf64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:58.274030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56782b-b355-4375-b792-7182ea08bf64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:58.275428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56782b-b355-4375-b792-7182ea08bf64 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:58.286853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 628/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:58.292573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d80f9b-5a3e-461c-a45e-af3d94ba3bf4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:58.324558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:58.391435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d80f9b-5a3e-461c-a45e-af3d94ba3bf4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:58.424792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce09e-96c4-45bb-90f2-4f5f870e4425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:58.425413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce09e-96c4-45bb-90f2-4f5f870e4425 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:58.425697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce09e-96c4-45bb-90f2-4f5f870e4425 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:58.425972 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce09e-96c4-45bb-90f2-4f5f870e4425 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:58.432448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ce09e-96c4-45bb-90f2-4f5f870e4425 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:58.433877 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ce09e-96c4-45bb-90f2-4f5f870e4425 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-400ce09e-96c4-45bb-90f2-4f5f870e4425 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:49:58.445031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:49:58.446072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:49:58.447358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:49:58.447358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:49:58.447358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:49:58.447358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:49:58.447358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:49:58.447358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:49:58.447358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:49:58.447358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:49:58.447358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 643/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:49:58.451937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56782b-b355-4375-b792-7182ea08bf64 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:58.509589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:58.551426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56782b-b355-4375-b792-7182ea08bf64 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:58.583510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430928b6-671b-453e-8da5-bb8bce500633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:58.584062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430928b6-671b-453e-8da5-bb8bce500633 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:58.584344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430928b6-671b-453e-8da5-bb8bce500633 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:58.584622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430928b6-671b-453e-8da5-bb8bce500633 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:58.585383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430928b6-671b-453e-8da5-bb8bce500633 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:49:58.585676 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430928b6-671b-453e-8da5-bb8bce500633 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:58.587121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430928b6-671b-453e-8da5-bb8bce500633 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:58.599114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 629/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:49:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:49:58.605187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce09e-96c4-45bb-90f2-4f5f870e4425 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:49:58.693264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:49:58.702277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce09e-96c4-45bb-90f2-4f5f870e4425 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:49:58.735023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02eec6-792b-47cb-ad66-1e776cf6681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:49:58.735662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02eec6-792b-47cb-ad66-1e776cf6681c None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:49:58.735940 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02eec6-792b-47cb-ad66-1e776cf6681c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:49:58.736213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02eec6-792b-47cb-ad66-1e776cf6681c None admin] PATH_INFO: `/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/551281a31ce24e2f9b81b36f69419311/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:49:58.738102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b02eec6-792b-47cb-ad66-1e776cf6681c None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7d94dcd9ec4eea9c5385b3050ff5df, user_id=551281a31ce24e2f9b81b36f69419311, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:49:58.739533 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b02eec6-792b-47cb-ad66-1e776cf6681c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:49:58.751474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 644/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jul 28 03:49:58 2022] PUT /identity/v3/projects/5c7d94dcd9ec4eea9c5385b3050ff5df/users/551281a31ce24e2f9b81b36f69419311/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:50:00.709755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5af3b-def6-4e64-ab97-55d3be652f11 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:00.710280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5af3b-def6-4e64-ab97-55d3be652f11 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:00.710369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5af3b-def6-4e64-ab97-55d3be652f11 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:00.711699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 630/1274] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:50:00.717547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593ddad-366a-4a03-aed0-2ff6e686d3b4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:00.717979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593ddad-366a-4a03-aed0-2ff6e686d3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:00.718116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593ddad-366a-4a03-aed0-2ff6e686d3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:00.736693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8593ddad-366a-4a03-aed0-2ff6e686d3b4 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:50:00.737882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8593ddad-366a-4a03-aed0-2ff6e686d3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:50:00.802916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 645/1275] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:50:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:50:00.827046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5af3b-def6-4e64-ab97-55d3be652f11 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:00.923465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5af3b-def6-4e64-ab97-55d3be652f11 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:00.940448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:00.957317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd567844-8f40-4bec-a673-13dbfa1fa8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:00.957941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd567844-8f40-4bec-a673-13dbfa1fa8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:00.958219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd567844-8f40-4bec-a673-13dbfa1fa8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:00.958486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd567844-8f40-4bec-a673-13dbfa1fa8e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:00.969532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd567844-8f40-4bec-a673-13dbfa1fa8e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-dd567844-8f40-4bec-a673-13dbfa1fa8e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 03:50:00.974292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:50:00.975392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:00.976351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 631/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jul 28 03:50:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:50:00.981516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593ddad-366a-4a03-aed0-2ff6e686d3b4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:01.062652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:01.081307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593ddad-366a-4a03-aed0-2ff6e686d3b4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:01.114335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d804d8-1bd0-41fb-b6f5-79bff73f859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:01.114971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d804d8-1bd0-41fb-b6f5-79bff73f859d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:01.115254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d804d8-1bd0-41fb-b6f5-79bff73f859d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:01.115527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d804d8-1bd0-41fb-b6f5-79bff73f859d None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:01.116239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d804d8-1bd0-41fb-b6f5-79bff73f859d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:50:01.116541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d804d8-1bd0-41fb-b6f5-79bff73f859d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:01.117895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d804d8-1bd0-41fb-b6f5-79bff73f859d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:01.126375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 646/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jul 28 03:50:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:50:01.132128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd567844-8f40-4bec-a673-13dbfa1fa8e9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:01.227841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd567844-8f40-4bec-a673-13dbfa1fa8e9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:01.261310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64daaf5-1b45-4f4a-b8e3-2c274588304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:01.261938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64daaf5-1b45-4f4a-b8e3-2c274588304b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:01.262224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64daaf5-1b45-4f4a-b8e3-2c274588304b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:01.262509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64daaf5-1b45-4f4a-b8e3-2c274588304b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:01.272293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64daaf5-1b45-4f4a-b8e3-2c274588304b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:01.273714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64daaf5-1b45-4f4a-b8e3-2c274588304b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b64daaf5-1b45-4f4a-b8e3-2c274588304b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:50:01.287945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:50:01.288965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:50:01.290076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:50:01.290076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:50:01.290076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:50:01.290076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:01.290076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 632/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:50:01.295392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d804d8-1bd0-41fb-b6f5-79bff73f859d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:01.393535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d804d8-1bd0-41fb-b6f5-79bff73f859d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:01.427099 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986de085-bd73-4336-aed8-827526d2445e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:01.427734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986de085-bd73-4336-aed8-827526d2445e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:01.428017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986de085-bd73-4336-aed8-827526d2445e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:01.428291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986de085-bd73-4336-aed8-827526d2445e None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:01.428977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986de085-bd73-4336-aed8-827526d2445e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:50:01.429281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986de085-bd73-4336-aed8-827526d2445e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:01.430635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986de085-bd73-4336-aed8-827526d2445e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:01.442205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 647/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 03:50:01.447938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64daaf5-1b45-4f4a-b8e3-2c274588304b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:01.478057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:01.547362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64daaf5-1b45-4f4a-b8e3-2c274588304b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:01.580046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dffa8-75a7-44d7-a9ab-5d7d2fd6d30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:01.580679 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dffa8-75a7-44d7-a9ab-5d7d2fd6d30a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:01.580963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dffa8-75a7-44d7-a9ab-5d7d2fd6d30a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:01.581240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dffa8-75a7-44d7-a9ab-5d7d2fd6d30a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:01.588509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dffa8-75a7-44d7-a9ab-5d7d2fd6d30a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:01.589960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dffa8-75a7-44d7-a9ab-5d7d2fd6d30a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b54dffa8-75a7-44d7-a9ab-5d7d2fd6d30a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 03:50:01.601463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:50:01.602538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:50:01.603565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:50:01.603565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:50:01.603565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 03:50:01.603565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 03:50:01.603565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 03:50:01.603565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 03:50:01.603565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 03:50:01.603565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 03:50:01.603565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 03:50:01.603565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:01.603565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 633/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:50:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:50:01.609046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986de085-bd73-4336-aed8-827526d2445e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:01.612813 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:01.706292 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986de085-bd73-4336-aed8-827526d2445e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:01.738542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521629c8-4170-4b62-8b9e-0ed251e1229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:01.739180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521629c8-4170-4b62-8b9e-0ed251e1229a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:01.739461 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521629c8-4170-4b62-8b9e-0ed251e1229a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:01.739735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521629c8-4170-4b62-8b9e-0ed251e1229a None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:01.740525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521629c8-4170-4b62-8b9e-0ed251e1229a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:50:01.740828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521629c8-4170-4b62-8b9e-0ed251e1229a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:01.742246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521629c8-4170-4b62-8b9e-0ed251e1229a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:01.759204 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 648/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jul 28 03:50:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:50:01.765896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dffa8-75a7-44d7-a9ab-5d7d2fd6d30a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:01.855344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:01.865127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dffa8-75a7-44d7-a9ab-5d7d2fd6d30a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:01.898116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04fcaa-a4f1-4c00-b66b-3c5e7dfb215c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:01.898724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04fcaa-a4f1-4c00-b66b-3c5e7dfb215c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:01.899050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04fcaa-a4f1-4c00-b66b-3c5e7dfb215c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:01.899325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04fcaa-a4f1-4c00-b66b-3c5e7dfb215c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:01.964590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04fcaa-a4f1-4c00-b66b-3c5e7dfb215c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:01.966718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04fcaa-a4f1-4c00-b66b-3c5e7dfb215c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:02.050793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:02.061858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 634/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jul 28 03:50:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 03:50:02.067604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521629c8-4170-4b62-8b9e-0ed251e1229a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:02.127538 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:02.165193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521629c8-4170-4b62-8b9e-0ed251e1229a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:02.198022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9938d1-ad8f-4bee-9b54-a9d57b5898ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:02.198650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9938d1-ad8f-4bee-9b54-a9d57b5898ec None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:02.198963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9938d1-ad8f-4bee-9b54-a9d57b5898ec None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:02.199243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9938d1-ad8f-4bee-9b54-a9d57b5898ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:02.208967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9938d1-ad8f-4bee-9b54-a9d57b5898ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:02.210408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9938d1-ad8f-4bee-9b54-a9d57b5898ec None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-7f9938d1-ad8f-4bee-9b54-a9d57b5898ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 03:50:02.225162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:50:02.226203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:50:02.227185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 03:50:02.227185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 03:50:02.227185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 03:50:02.227185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:50:02.227185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 649/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:50:02.231936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04fcaa-a4f1-4c00-b66b-3c5e7dfb215c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:02.328545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04fcaa-a4f1-4c00-b66b-3c5e7dfb215c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:02.361375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c39a6-a23f-4dc4-a84c-1f45967bb086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:02.361976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c39a6-a23f-4dc4-a84c-1f45967bb086 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:02.362260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c39a6-a23f-4dc4-a84c-1f45967bb086 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:02.362534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c39a6-a23f-4dc4-a84c-1f45967bb086 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:02.363259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c39a6-a23f-4dc4-a84c-1f45967bb086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:50:02.363556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c39a6-a23f-4dc4-a84c-1f45967bb086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:02.364991 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c39a6-a23f-4dc4-a84c-1f45967bb086 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:02.376551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 635/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jul 28 03:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:50:02.382026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9938d1-ad8f-4bee-9b54-a9d57b5898ec None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:02.483198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9938d1-ad8f-4bee-9b54-a9d57b5898ec None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:02.495977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:02.517598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571da6f-2e83-490b-aff8-affa86e77378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:02.518230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571da6f-2e83-490b-aff8-affa86e77378 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:02.518512 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571da6f-2e83-490b-aff8-affa86e77378 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:02.518805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571da6f-2e83-490b-aff8-affa86e77378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:02.525196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571da6f-2e83-490b-aff8-affa86e77378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:02.526617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571da6f-2e83-490b-aff8-affa86e77378 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1571da6f-2e83-490b-aff8-affa86e77378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 03:50:02.538332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:50:02.539379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:50:02.540541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 03:50:02.540541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 03:50:02.540541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 03:50:02.540541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 03:50:02.540541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 03:50:02.540541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 03:50:02.540541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 03:50:02.540541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 28 03:50:02.540541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 650/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jul 28 03:50:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:50:02.546211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c39a6-a23f-4dc4-a84c-1f45967bb086 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:02.595222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:02.645259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c39a6-a23f-4dc4-a84c-1f45967bb086 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:02.677860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764923f-70da-4527-a79c-4fb74926c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:02.678467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764923f-70da-4527-a79c-4fb74926c53c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:02.678754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764923f-70da-4527-a79c-4fb74926c53c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:02.679065 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764923f-70da-4527-a79c-4fb74926c53c None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:02.679811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764923f-70da-4527-a79c-4fb74926c53c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:50:02.680120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764923f-70da-4527-a79c-4fb74926c53c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:02.681560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764923f-70da-4527-a79c-4fb74926c53c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:02.693946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 636/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jul 28 03:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:50:02.699946 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571da6f-2e83-490b-aff8-affa86e77378 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:02.796660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571da6f-2e83-490b-aff8-affa86e77378 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:02.829775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df60f7-fb11-4c18-9696-390b1cb48ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:02.830371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df60f7-fb11-4c18-9696-390b1cb48ece None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:02.830652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df60f7-fb11-4c18-9696-390b1cb48ece None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:02.830953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df60f7-fb11-4c18-9696-390b1cb48ece None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:02.831731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df60f7-fb11-4c18-9696-390b1cb48ece None admin] RBAC: Adding query filter params (role.id=a37ae24c516b48c69e13721c013134a2, scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df, user.id=551281a31ce24e2f9b81b36f69419311) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:50:02.832041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df60f7-fb11-4c18-9696-390b1cb48ece None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:02.833454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df60f7-fb11-4c18-9696-390b1cb48ece None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:02.839100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 651/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jul 28 03:50:02 2022] GET /identity/v3/role_assignments?user.id=551281a31ce24e2f9b81b36f69419311&scope.project.id=5c7d94dcd9ec4eea9c5385b3050ff5df&role.id=a37ae24c516b48c69e13721c013134a2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:50:04.819213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a7411-32cc-4077-8e36-85118fca1ebb None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:04.819779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a7411-32cc-4077-8e36-85118fca1ebb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:04.819779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a7411-32cc-4077-8e36-85118fca1ebb None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:04.821096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 637/1288] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:50:04.826680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e87b-9dc3-47ec-aec8-3edb39abc5e7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:04.827095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e87b-9dc3-47ec-aec8-3edb39abc5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:04.827259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e87b-9dc3-47ec-aec8-3edb39abc5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:04.845486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c769e87b-9dc3-47ec-aec8-3edb39abc5e7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:50:04.846660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c769e87b-9dc3-47ec-aec8-3edb39abc5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:50:04.875205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:04.912302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 652/1289] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:50:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:50:04.933487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a7411-32cc-4077-8e36-85118fca1ebb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:05.035582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a7411-32cc-4077-8e36-85118fca1ebb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:05.046515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:05.070336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975d6ca-7ebc-4057-b039-92acdd724414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:05.070947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975d6ca-7ebc-4057-b039-92acdd724414 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:05.071233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975d6ca-7ebc-4057-b039-92acdd724414 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:05.071508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975d6ca-7ebc-4057-b039-92acdd724414 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:05.072135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975d6ca-7ebc-4057-b039-92acdd724414 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:05.073509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975d6ca-7ebc-4057-b039-92acdd724414 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-0975d6ca-7ebc-4057-b039-92acdd724414 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:50:05.082454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:50:05.083754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:05.084902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 638/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jul 28 03:50:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:50:05.089768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769e87b-9dc3-47ec-aec8-3edb39abc5e7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:05.187046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769e87b-9dc3-47ec-aec8-3edb39abc5e7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:05.220426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159393c6-3bce-4643-a2fe-4d47f0e86043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:05.221050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159393c6-3bce-4643-a2fe-4d47f0e86043 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:05.221327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159393c6-3bce-4643-a2fe-4d47f0e86043 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:05.221599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159393c6-3bce-4643-a2fe-4d47f0e86043 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:05.222272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159393c6-3bce-4643-a2fe-4d47f0e86043 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:50:05.222574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159393c6-3bce-4643-a2fe-4d47f0e86043 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:05.223961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159393c6-3bce-4643-a2fe-4d47f0e86043 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:05.228592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 653/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Jul 28 03:50:05 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:50:05.234845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975d6ca-7ebc-4057-b039-92acdd724414 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:05.342290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975d6ca-7ebc-4057-b039-92acdd724414 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:05.375339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443a55b-4c17-4063-b051-476433be71f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:05.375948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443a55b-4c17-4063-b051-476433be71f8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:05.376235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443a55b-4c17-4063-b051-476433be71f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:05.376514 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443a55b-4c17-4063-b051-476433be71f8 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:05.377192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443a55b-4c17-4063-b051-476433be71f8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:50:05.377499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443a55b-4c17-4063-b051-476433be71f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:05.378914 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443a55b-4c17-4063-b051-476433be71f8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:05.383255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 639/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Jul 28 03:50:05 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:50:07.397885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113f88e-0671-45a8-a768-ef02d4d9cb56 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:07.398433 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113f88e-0671-45a8-a768-ef02d4d9cb56 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:07.398433 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113f88e-0671-45a8-a768-ef02d4d9cb56 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:07.399806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 654/1293] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:50:07.405780 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa186-9353-4420-bc0f-0757823fd2b1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:07.406144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa186-9353-4420-bc0f-0757823fd2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:07.406341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa186-9353-4420-bc0f-0757823fd2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:07.424728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-482aa186-9353-4420-bc0f-0757823fd2b1 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:50:07.425907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-482aa186-9353-4420-bc0f-0757823fd2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:50:07.455929 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:07.492674 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 640/1294] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:50:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:50:07.516037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113f88e-0671-45a8-a768-ef02d4d9cb56 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:07.548811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:07.618725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113f88e-0671-45a8-a768-ef02d4d9cb56 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:07.651090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056794c-cbfe-40e4-85e1-c4d731e92af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:07.651642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056794c-cbfe-40e4-85e1-c4d731e92af1 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:07.651927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056794c-cbfe-40e4-85e1-c4d731e92af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:07.652203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056794c-cbfe-40e4-85e1-c4d731e92af1 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:07.653413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056794c-cbfe-40e4-85e1-c4d731e92af1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:07.654777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056794c-cbfe-40e4-85e1-c4d731e92af1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:07.669712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 655/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jul 28 03:50:07 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:50:09.694629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c177c-d9d0-46de-948b-aa71f3ce1f44 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:09.695186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c177c-d9d0-46de-948b-aa71f3ce1f44 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:09.695291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c177c-d9d0-46de-948b-aa71f3ce1f44 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:09.696583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 641/1296] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:50:09.702080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fa306-e1b3-4b9b-abf9-04701869aff8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:09.702462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fa306-e1b3-4b9b-abf9-04701869aff8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:09.702638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fa306-e1b3-4b9b-abf9-04701869aff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:09.720664 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-503fa306-e1b3-4b9b-abf9-04701869aff8 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:50:09.721845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-503fa306-e1b3-4b9b-abf9-04701869aff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:50:09.791819 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 656/1297] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:50:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:50:09.814574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c177c-d9d0-46de-948b-aa71f3ce1f44 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:09.911585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c177c-d9d0-46de-948b-aa71f3ce1f44 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:09.949786 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c206f66-41c4-4102-901c-b01a17ba38ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:09.950403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c206f66-41c4-4102-901c-b01a17ba38ee None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:09.950683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c206f66-41c4-4102-901c-b01a17ba38ee None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:09.950987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c206f66-41c4-4102-901c-b01a17ba38ee None admin] PATH_INFO: `/v3/services/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:09.951600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c206f66-41c4-4102-901c-b01a17ba38ee None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:09.952948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c206f66-41c4-4102-901c-b01a17ba38ee None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-4c206f66-41c4-4102-901c-b01a17ba38ee None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:50:09.961500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:50:09.962658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:09.963894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 642/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jul 28 03:50:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:50:09.969961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fa306-e1b3-4b9b-abf9-04701869aff8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:09.988784 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:10.070699 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fa306-e1b3-4b9b-abf9-04701869aff8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:10.105466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc34c6-4b78-45e5-98c2-909bf33e88f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:10.106515 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc34c6-4b78-45e5-98c2-909bf33e88f0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:10.106827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc34c6-4b78-45e5-98c2-909bf33e88f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:10.107111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc34c6-4b78-45e5-98c2-909bf33e88f0 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:10.107796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc34c6-4b78-45e5-98c2-909bf33e88f0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:50:10.108101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc34c6-4b78-45e5-98c2-909bf33e88f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:10.109457 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc34c6-4b78-45e5-98c2-909bf33e88f0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:10.114456 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 657/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Jul 28 03:50:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:50:10.120362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c206f66-41c4-4102-901c-b01a17ba38ee None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:10.151462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:10.218486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c206f66-41c4-4102-901c-b01a17ba38ee None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:10.253075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece43523-2ff9-4c8b-be74-4465c330efa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:10.253686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece43523-2ff9-4c8b-be74-4465c330efa5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:10.253973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece43523-2ff9-4c8b-be74-4465c330efa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:10.254245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece43523-2ff9-4c8b-be74-4465c330efa5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:10.255040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece43523-2ff9-4c8b-be74-4465c330efa5 None admin] RBAC: Adding query filter params (interface=public, service_id=91e3c1eb850c48ef97f33ff6e608c686, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:50:10.255360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece43523-2ff9-4c8b-be74-4465c330efa5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:10.256779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece43523-2ff9-4c8b-be74-4465c330efa5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:10.261904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 643/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jul 28 03:50:10 2022] GET /identity/v3/endpoints?service_id=91e3c1eb850c48ef97f33ff6e608c686&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:50:10.269387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc34c6-4b78-45e5-98c2-909bf33e88f0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:10.353444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:10.370948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc34c6-4b78-45e5-98c2-909bf33e88f0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:10.404054 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b738efe-f95d-4628-8150-80e93d12e570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:10.404656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b738efe-f95d-4628-8150-80e93d12e570 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:10.404935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b738efe-f95d-4628-8150-80e93d12e570 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:10.405205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b738efe-f95d-4628-8150-80e93d12e570 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:10.405848 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b738efe-f95d-4628-8150-80e93d12e570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:10.407232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b738efe-f95d-4628-8150-80e93d12e570 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:10.414199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 658/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jul 28 03:50:10 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:50:12.380971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d90bfe-5101-4150-a69c-5dd042e2ce48 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:12.381534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d90bfe-5101-4150-a69c-5dd042e2ce48 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:12.381534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d90bfe-5101-4150-a69c-5dd042e2ce48 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:12.382972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 644/1302] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:50:12.388602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9f93-8cab-40a1-906a-3016151b1331 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:12.389000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9f93-8cab-40a1-906a-3016151b1331 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:12.389141 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9f93-8cab-40a1-906a-3016151b1331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:12.407513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1abd9f93-8cab-40a1-906a-3016151b1331 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:50:12.408702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1abd9f93-8cab-40a1-906a-3016151b1331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:50:12.473087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 659/1303] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:50:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 03:50:12.494865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d90bfe-5101-4150-a69c-5dd042e2ce48 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:12.588308 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:12.595760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d90bfe-5101-4150-a69c-5dd042e2ce48 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:12.628348 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598cd46-b57a-43f2-922e-b56520613edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:12.628973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598cd46-b57a-43f2-922e-b56520613edc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:12.629251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598cd46-b57a-43f2-922e-b56520613edc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:12.629520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598cd46-b57a-43f2-922e-b56520613edc None admin] PATH_INFO: `/v3/services/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:12.630134 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598cd46-b57a-43f2-922e-b56520613edc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:12.631559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598cd46-b57a-43f2-922e-b56520613edc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-4598cd46-b57a-43f2-922e-b56520613edc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 03:50:12.640551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 03:50:12.641711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 28 03:50:12.642742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 645/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jul 28 03:50:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 03:50:12.647675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd9f93-8cab-40a1-906a-3016151b1331 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:12.721459 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:12.743908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd9f93-8cab-40a1-906a-3016151b1331 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:12.777366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012edeb-09ad-4a5c-aa86-9678a3b9fc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:12.777968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012edeb-09ad-4a5c-aa86-9678a3b9fc86 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:12.778246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012edeb-09ad-4a5c-aa86-9678a3b9fc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:12.778516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012edeb-09ad-4a5c-aa86-9678a3b9fc86 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:12.779210 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012edeb-09ad-4a5c-aa86-9678a3b9fc86 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:50:12.779509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012edeb-09ad-4a5c-aa86-9678a3b9fc86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:12.780898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012edeb-09ad-4a5c-aa86-9678a3b9fc86 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:12.786114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 660/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Jul 28 03:50:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:50:12.791733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598cd46-b57a-43f2-922e-b56520613edc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:50:12.887612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598cd46-b57a-43f2-922e-b56520613edc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:50:12.919223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e1ab9-fbe5-4697-bfb9-45d14a7ec3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:50:12.919817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1ab9-fbe5-4697-bfb9-45d14a7ec3d1 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:50:12.920093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1ab9-fbe5-4697-bfb9-45d14a7ec3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:50:12.920361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1ab9-fbe5-4697-bfb9-45d14a7ec3d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:50:12.921059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e1ab9-fbe5-4697-bfb9-45d14a7ec3d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:50:12.922398 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e1ab9-fbe5-4697-bfb9-45d14a7ec3d1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:50:12.941349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 646/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:50:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:50:22.782979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=30 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:50:22.937072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:16.106309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac81210-e20e-4353-8f76-0788a889c044 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:16.106925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac81210-e20e-4353-8f76-0788a889c044 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:16.106925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac81210-e20e-4353-8f76-0788a889c044 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:16.108412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 661/1307] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:16.113422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c496b4-434e-4a95-a946-a910a834fd01 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:16.113848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c496b4-434e-4a95-a946-a910a834fd01 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:16.114008 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c496b4-434e-4a95-a946-a910a834fd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:16.245625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-43c496b4-434e-4a95-a946-a910a834fd01 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:16.247227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-43c496b4-434e-4a95-a946-a910a834fd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:16.455263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 647/1308] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:16.539947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60285-621b-45eb-9856-11ab07b9b45d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:16.540243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60285-621b-45eb-9856-11ab07b9b45d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:16.540772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60285-621b-45eb-9856-11ab07b9b45d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:16.542883 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 662/1309] 10.222.0.46 () {54 vars in 965 bytes} [Thu Jul 28 03:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:16.549808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79b6e3-8ab4-47f4-97ca-2ac69f80886a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:16.550122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79b6e3-8ab4-47f4-97ca-2ac69f80886a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:16.550405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79b6e3-8ab4-47f4-97ca-2ac69f80886a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:16.708173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fb79b6e3-8ab4-47f4-97ca-2ac69f80886a None None] MFA Rules not processed for user `6e31408bfea4412d8f68aea9b3c7528d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:16.709618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fb79b6e3-8ab4-47f4-97ca-2ac69f80886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:16.760815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:16.850422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 648/1310] 10.222.0.46 () {58 vars in 1062 bytes} [Thu Jul 28 03:51:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:16.850998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c60285-621b-45eb-9856-11ab07b9b45d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:16.853325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e7c60285-621b-45eb-9856-11ab07b9b45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:16.993062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c60285-621b-45eb-9856-11ab07b9b45d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:17.097659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111f686-ab7b-4e96-b416-8867a98a4657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:17.098456 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111f686-ab7b-4e96-b416-8867a98a4657 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:17.098867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111f686-ab7b-4e96-b416-8867a98a4657 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:17.099246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111f686-ab7b-4e96-b416-8867a98a4657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:17.101050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a111f686-ab7b-4e96-b416-8867a98a4657 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:17.168395 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:17.235391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111f686-ab7b-4e96-b416-8867a98a4657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:17.237151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111f686-ab7b-4e96-b416-8867a98a4657 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:17.260875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 663/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 03:51:19.580874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7204f7-175e-4e04-afb5-30c8f47a8c23 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:19.581412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7204f7-175e-4e04-afb5-30c8f47a8c23 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:19.581412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7204f7-175e-4e04-afb5-30c8f47a8c23 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:19.584465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 649/1312] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:19.590687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65868304-7044-4a27-92f9-dbf429e82d6b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:19.591256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65868304-7044-4a27-92f9-dbf429e82d6b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:19.591256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65868304-7044-4a27-92f9-dbf429e82d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:19.671197 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-65868304-7044-4a27-92f9-dbf429e82d6b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:19.672330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-65868304-7044-4a27-92f9-dbf429e82d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:19.776142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 664/1313] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:19.815294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7204f7-175e-4e04-afb5-30c8f47a8c23 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:19.816894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ec7204f7-175e-4e04-afb5-30c8f47a8c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:19.923152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7204f7-175e-4e04-afb5-30c8f47a8c23 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:19.995598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:19.998919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb2c3e-4c51-462e-81ff-03af273d9d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:19.999952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb2c3e-4c51-462e-81ff-03af273d9d65 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:20.000237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb2c3e-4c51-462e-81ff-03af273d9d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:20.000513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb2c3e-4c51-462e-81ff-03af273d9d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:20.002165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e5cb2c3e-4c51-462e-81ff-03af273d9d65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:20.110367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb2c3e-4c51-462e-81ff-03af273d9d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:20.111834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb2c3e-4c51-462e-81ff-03af273d9d65 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:20.129727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 650/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:51:22.416149 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2630e40-70f7-4c26-890b-519ead2c0af2 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:22.416871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2630e40-70f7-4c26-890b-519ead2c0af2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:22.416871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2630e40-70f7-4c26-890b-519ead2c0af2 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:22.418044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 665/1315] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:22.424581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398f925-5225-4912-adb1-6e178426c098 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:22.424982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398f925-5225-4912-adb1-6e178426c098 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:22.425140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398f925-5225-4912-adb1-6e178426c098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:22.489310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9398f925-5225-4912-adb1-6e178426c098 None None] MFA Rules not processed for user `5993366070364e2ea23f57f39e958895`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:22.490525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9398f925-5225-4912-adb1-6e178426c098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:22.542717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:22.588194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 651/1316] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:22.628091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2630e40-70f7-4c26-890b-519ead2c0af2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:22.629615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d2630e40-70f7-4c26-890b-519ead2c0af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:22.640087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:22.731415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2630e40-70f7-4c26-890b-519ead2c0af2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:22.801846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd788bfd-5deb-48e4-80ff-fcc49e43dff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:22.802445 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788bfd-5deb-48e4-80ff-fcc49e43dff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:22.802722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788bfd-5deb-48e4-80ff-fcc49e43dff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:22.803016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788bfd-5deb-48e4-80ff-fcc49e43dff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:22.804616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dd788bfd-5deb-48e4-80ff-fcc49e43dff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:22.865561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:22.911300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd788bfd-5deb-48e4-80ff-fcc49e43dff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:22.912732 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd788bfd-5deb-48e4-80ff-fcc49e43dff8 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:22.931507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 666/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:22 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:51:25.502748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9849a-fcb4-428c-a393-1d2ad8e73aaf None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:25.503299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9849a-fcb4-428c-a393-1d2ad8e73aaf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:25.503299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9849a-fcb4-428c-a393-1d2ad8e73aaf None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:25.504663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 652/1318] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:25.510208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920bf7-06bb-4f4f-8ac2-9c5e1472915b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:25.510612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920bf7-06bb-4f4f-8ac2-9c5e1472915b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:25.510769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920bf7-06bb-4f4f-8ac2-9c5e1472915b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:25.577281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ee920bf7-06bb-4f4f-8ac2-9c5e1472915b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:25.578471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ee920bf7-06bb-4f4f-8ac2-9c5e1472915b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:25.674346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 667/1319] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:25.711758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9849a-fcb4-428c-a393-1d2ad8e73aaf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:25.713813 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-19d9849a-fcb4-428c-a393-1d2ad8e73aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:25.815968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9849a-fcb4-428c-a393-1d2ad8e73aaf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:25.889573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adc7dd-5e43-4de7-8947-70928ec989dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:25.890182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adc7dd-5e43-4de7-8947-70928ec989dc service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:25.890464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adc7dd-5e43-4de7-8947-70928ec989dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:25.890741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adc7dd-5e43-4de7-8947-70928ec989dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:25.892417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-81adc7dd-5e43-4de7-8947-70928ec989dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:25.900229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:25.999603 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81adc7dd-5e43-4de7-8947-70928ec989dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:26.001048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81adc7dd-5e43-4de7-8947-70928ec989dc service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:26.019332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 653/1320] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:51:28.658035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf931871-df71-44e8-8d41-dd86d00f6cd5 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:28.658563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf931871-df71-44e8-8d41-dd86d00f6cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:28.658653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf931871-df71-44e8-8d41-dd86d00f6cd5 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:28.660255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 668/1321] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:28.666454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d11c7-9250-4008-acca-0a093f9bc38a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:28.666909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d11c7-9250-4008-acca-0a093f9bc38a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:28.667041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d11c7-9250-4008-acca-0a093f9bc38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:28.732385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-239d11c7-9250-4008-acca-0a093f9bc38a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:28.733585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-239d11c7-9250-4008-acca-0a093f9bc38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:28.791387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:28.831276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 654/1322] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:28.881330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e51a7-e11a-4bcb-ae86-94ba2fbfead8 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:28.881617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e51a7-e11a-4bcb-ae86-94ba2fbfead8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:28.881899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e51a7-e11a-4bcb-ae86-94ba2fbfead8 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:28.883361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 669/1323] 10.222.0.46 () {54 vars in 965 bytes} [Thu Jul 28 03:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:28.890338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf230a-d85e-4615-b348-b17f570ade7a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:28.890633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf230a-d85e-4615-b348-b17f570ade7a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:28.890949 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf230a-d85e-4615-b348-b17f570ade7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:28.974390 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bcdf230a-d85e-4615-b348-b17f570ade7a None None] MFA Rules not processed for user `6e31408bfea4412d8f68aea9b3c7528d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:28.975673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bcdf230a-d85e-4615-b348-b17f570ade7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:29.057099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 655/1324] 10.222.0.46 () {58 vars in 1062 bytes} [Thu Jul 28 03:51:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:29.066182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e51a7-e11a-4bcb-ae86-94ba2fbfead8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:29.067762 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d29e51a7-e11a-4bcb-ae86-94ba2fbfead8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:29.088265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:29.172960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e51a7-e11a-4bcb-ae86-94ba2fbfead8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:29.246484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07808c-f719-43d0-b210-a96c18afa87b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:29.247133 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07808c-f719-43d0-b210-a96c18afa87b service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:29.247452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07808c-f719-43d0-b210-a96c18afa87b service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:29.247806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07808c-f719-43d0-b210-a96c18afa87b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:29.249563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8d07808c-f719-43d0-b210-a96c18afa87b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:29.321704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:29.360432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07808c-f719-43d0-b210-a96c18afa87b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:29.361907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07808c-f719-43d0-b210-a96c18afa87b service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:29.381672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 670/1325] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:51:31.651101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac77f2-52b4-4420-b977-792f66becedc None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:31.651627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac77f2-52b4-4420-b977-792f66becedc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:31.651716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac77f2-52b4-4420-b977-792f66becedc None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:31.653017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 656/1326] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:31.658144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb18da9-4b2c-4c65-a2c7-37aa58b13e93 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:31.658563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb18da9-4b2c-4c65-a2c7-37aa58b13e93 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:31.658841 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb18da9-4b2c-4c65-a2c7-37aa58b13e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:31.725018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5eb18da9-4b2c-4c65-a2c7-37aa58b13e93 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:31.726212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5eb18da9-4b2c-4c65-a2c7-37aa58b13e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:31.825573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 671/1327] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:31.861262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac77f2-52b4-4420-b977-792f66becedc None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:31.862927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-afac77f2-52b4-4420-b977-792f66becedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:31.924520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:31.973823 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac77f2-52b4-4420-b977-792f66becedc None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:32.049882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03951c-c539-4917-9f96-ebbda192e1de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:32.050504 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03951c-c539-4917-9f96-ebbda192e1de service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:32.050810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03951c-c539-4917-9f96-ebbda192e1de service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:32.051081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03951c-c539-4917-9f96-ebbda192e1de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:32.052700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ef03951c-c539-4917-9f96-ebbda192e1de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:32.158435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:32.161218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03951c-c539-4917-9f96-ebbda192e1de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:32.163316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03951c-c539-4917-9f96-ebbda192e1de service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:32.181871 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 657/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:51:34.242960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b381f63-285b-487c-95eb-08015041155f None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:34.243607 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b381f63-285b-487c-95eb-08015041155f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:34.243705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b381f63-285b-487c-95eb-08015041155f None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:34.246171 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 672/1329] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:34.251607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fee0e5-16bc-42dd-ac9c-1709bb805b9c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:34.252009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fee0e5-16bc-42dd-ac9c-1709bb805b9c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:34.252153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fee0e5-16bc-42dd-ac9c-1709bb805b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:34.317512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-74fee0e5-16bc-42dd-ac9c-1709bb805b9c None None] MFA Rules not processed for user `5993366070364e2ea23f57f39e958895`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:34.318714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-74fee0e5-16bc-42dd-ac9c-1709bb805b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:34.414151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 658/1330] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:34.453153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b381f63-285b-487c-95eb-08015041155f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:34.454707 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4b381f63-285b-487c-95eb-08015041155f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:34.485662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:34.562535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b381f63-285b-487c-95eb-08015041155f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:34.651631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95621985-5108-4c65-a480-2898f8d3df58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:34.652268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95621985-5108-4c65-a480-2898f8d3df58 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:34.652554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95621985-5108-4c65-a480-2898f8d3df58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:34.652831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95621985-5108-4c65-a480-2898f8d3df58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:34.654512 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-95621985-5108-4c65-a480-2898f8d3df58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:34.739840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:34.771740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95621985-5108-4c65-a480-2898f8d3df58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:34.773180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95621985-5108-4c65-a480-2898f8d3df58 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:34.792624 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 673/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:34 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:51:37.224366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf240ffa-029d-48d9-a0dc-4da318487f0f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:37.224954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf240ffa-029d-48d9-a0dc-4da318487f0f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:37.224954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf240ffa-029d-48d9-a0dc-4da318487f0f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:37.226288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 659/1332] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:37.231513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b0a3-0c49-4d45-a79a-ff35f6471d49 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:37.231965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b0a3-0c49-4d45-a79a-ff35f6471d49 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:37.232066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b0a3-0c49-4d45-a79a-ff35f6471d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:37.298837 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5094b0a3-0c49-4d45-a79a-ff35f6471d49 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:37.300027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5094b0a3-0c49-4d45-a79a-ff35f6471d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:37.395422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 674/1333] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:37.432083 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf240ffa-029d-48d9-a0dc-4da318487f0f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:37.433632 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bf240ffa-029d-48d9-a0dc-4da318487f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:37.502141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:37.537081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf240ffa-029d-48d9-a0dc-4da318487f0f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:37.609912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09c513-a697-4076-a350-3629c86238d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:37.610528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09c513-a697-4076-a350-3629c86238d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:37.610840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09c513-a697-4076-a350-3629c86238d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:37.611117 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09c513-a697-4076-a350-3629c86238d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:37.612775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-da09c513-a697-4076-a350-3629c86238d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:37.719320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09c513-a697-4076-a350-3629c86238d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:37.720756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09c513-a697-4076-a350-3629c86238d0 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:37.731676 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:37.739816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 660/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 03:51:39.762814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d22f38-66aa-40ad-9f0c-6f2ec72ab916 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:39.763367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d22f38-66aa-40ad-9f0c-6f2ec72ab916 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:39.763367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d22f38-66aa-40ad-9f0c-6f2ec72ab916 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:39.764733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 675/1335] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:39.769709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c322ad1-7c96-4534-b7c6-e4f51bd7a312 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:39.770139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c322ad1-7c96-4534-b7c6-e4f51bd7a312 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:39.770243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c322ad1-7c96-4534-b7c6-e4f51bd7a312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:39.835120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9c322ad1-7c96-4534-b7c6-e4f51bd7a312 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:39.836312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9c322ad1-7c96-4534-b7c6-e4f51bd7a312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:39.928996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 661/1336] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:39.968574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d22f38-66aa-40ad-9f0c-6f2ec72ab916 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:39.970100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a3d22f38-66aa-40ad-9f0c-6f2ec72ab916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:40.010176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:40.073518 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d22f38-66aa-40ad-9f0c-6f2ec72ab916 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:40.144106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf011a79-9d4b-4906-a323-fbc6513664fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:40.144649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf011a79-9d4b-4906-a323-fbc6513664fb service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:40.144928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf011a79-9d4b-4906-a323-fbc6513664fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:40.145200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf011a79-9d4b-4906-a323-fbc6513664fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:40.146825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bf011a79-9d4b-4906-a323-fbc6513664fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:40.228812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:40.254392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf011a79-9d4b-4906-a323-fbc6513664fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:40.255887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf011a79-9d4b-4906-a323-fbc6513664fb service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:40.273825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 676/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:51:42.618715 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178479f-a033-4f99-a205-ec59f3aa2f60 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:42.619440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178479f-a033-4f99-a205-ec59f3aa2f60 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:42.619440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178479f-a033-4f99-a205-ec59f3aa2f60 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:42.620648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 662/1338] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:42.625713 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a434f-586d-4c46-9fcb-c5a39979dbdf None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:42.626142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a434f-586d-4c46-9fcb-c5a39979dbdf None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:42.626271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a434f-586d-4c46-9fcb-c5a39979dbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:42.691629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-be5a434f-586d-4c46-9fcb-c5a39979dbdf None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:42.692830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-be5a434f-586d-4c46-9fcb-c5a39979dbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:42.788338 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 677/1339] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:42.824747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178479f-a033-4f99-a205-ec59f3aa2f60 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:42.826267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d178479f-a033-4f99-a205-ec59f3aa2f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:42.900210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:42.927927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178479f-a033-4f99-a205-ec59f3aa2f60 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:43.001224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461e650-9169-4c01-827d-63ccfd938502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:43.001926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461e650-9169-4c01-827d-63ccfd938502 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:43.002212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461e650-9169-4c01-827d-63ccfd938502 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:43.002487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461e650-9169-4c01-827d-63ccfd938502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:43.004236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2461e650-9169-4c01-827d-63ccfd938502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:43.115434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461e650-9169-4c01-827d-63ccfd938502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:43.116867 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461e650-9169-4c01-827d-63ccfd938502 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:43.135846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 663/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:51:47.186175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dc069-ac73-422a-b180-526ad2ef880d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:47.186724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dc069-ac73-422a-b180-526ad2ef880d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:47.186724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dc069-ac73-422a-b180-526ad2ef880d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:47.188166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 678/1341] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:47.193687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3959d-dae7-47a0-b7e3-36fce1358e65 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:47.194100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3959d-dae7-47a0-b7e3-36fce1358e65 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:47.194329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3959d-dae7-47a0-b7e3-36fce1358e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:47.200242 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:47.263218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bdd3959d-dae7-47a0-b7e3-36fce1358e65 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:47.264433 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bdd3959d-dae7-47a0-b7e3-36fce1358e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:47.362829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 664/1342] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:47.400968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dc069-ac73-422a-b180-526ad2ef880d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:47.402517 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-277dc069-ac73-422a-b180-526ad2ef880d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:47.452560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:47.507784 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dc069-ac73-422a-b180-526ad2ef880d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:47.584620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8380c-0206-4142-ba36-f273966873b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:47.585240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8380c-0206-4142-ba36-f273966873b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:47.585524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8380c-0206-4142-ba36-f273966873b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:47.585799 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8380c-0206-4142-ba36-f273966873b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:47.588170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-98c8380c-0206-4142-ba36-f273966873b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:47.681946 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:47.699336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8380c-0206-4142-ba36-f273966873b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:47.700797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8380c-0206-4142-ba36-f273966873b7 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:47.720306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 679/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:51:50.402717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff4641-1df7-46c5-910a-11a9e632cdb4 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:50.403289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff4641-1df7-46c5-910a-11a9e632cdb4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:50.403289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff4641-1df7-46c5-910a-11a9e632cdb4 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:50.404681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 665/1344] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:50.410672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed065b-c59a-4902-a174-447abdf735e6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:50.411209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed065b-c59a-4902-a174-447abdf735e6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:50.411300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed065b-c59a-4902-a174-447abdf735e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:50.492746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-69ed065b-c59a-4902-a174-447abdf735e6 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:50.494528 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-69ed065b-c59a-4902-a174-447abdf735e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:50.595472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 680/1345] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:50.633533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff4641-1df7-46c5-910a-11a9e632cdb4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:50.635604 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c1ff4641-1df7-46c5-910a-11a9e632cdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:50.722488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:50.741053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff4641-1df7-46c5-910a-11a9e632cdb4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:50.815252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a722fdc-50c9-4a4f-8b12-c045b83680d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:50.815873 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a722fdc-50c9-4a4f-8b12-c045b83680d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:50.816159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a722fdc-50c9-4a4f-8b12-c045b83680d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:50.816439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a722fdc-50c9-4a4f-8b12-c045b83680d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:50.818081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8a722fdc-50c9-4a4f-8b12-c045b83680d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:50.928034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a722fdc-50c9-4a4f-8b12-c045b83680d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:50.929497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a722fdc-50c9-4a4f-8b12-c045b83680d3 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:50.947364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 666/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:51:55.225251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43162eab-cdfd-4717-abd8-1eeaa814d3d2 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:55.225705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43162eab-cdfd-4717-abd8-1eeaa814d3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:55.225798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43162eab-cdfd-4717-abd8-1eeaa814d3d2 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:55.227272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 681/1347] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:51:55.232967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277c310-0712-4570-b82b-b0fdb5d3d781 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:55.233455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277c310-0712-4570-b82b-b0fdb5d3d781 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:55.233656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277c310-0712-4570-b82b-b0fdb5d3d781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:55.250609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:55.306563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d277c310-0712-4570-b82b-b0fdb5d3d781 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:51:55.307823 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d277c310-0712-4570-b82b-b0fdb5d3d781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:55.412148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 667/1348] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:51:55.450677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43162eab-cdfd-4717-abd8-1eeaa814d3d2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:51:55.452263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-43162eab-cdfd-4717-abd8-1eeaa814d3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:55.515085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:55.566996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43162eab-cdfd-4717-abd8-1eeaa814d3d2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:51:55.644782 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59aee7-0cea-410d-a02f-0da219b04888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:51:55.645424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59aee7-0cea-410d-a02f-0da219b04888 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:51:55.645702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59aee7-0cea-410d-a02f-0da219b04888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:51:55.645981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59aee7-0cea-410d-a02f-0da219b04888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:51:55.647685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8d59aee7-0cea-410d-a02f-0da219b04888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:51:55.760729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:51:55.764955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59aee7-0cea-410d-a02f-0da219b04888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:51:55.766418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59aee7-0cea-410d-a02f-0da219b04888 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:51:55.787372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 682/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:51:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:00.684812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14886c55-41d2-4fac-a7da-a6dea5d4e917 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:00.685278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14886c55-41d2-4fac-a7da-a6dea5d4e917 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:00.685380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14886c55-41d2-4fac-a7da-a6dea5d4e917 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:00.687177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 668/1350] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jul 28 03:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:00.694328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25033f5-dc27-4af3-ad15-fc740bdc1657 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:00.694733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25033f5-dc27-4af3-ad15-fc740bdc1657 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:00.695099 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25033f5-dc27-4af3-ad15-fc740bdc1657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:00.826847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e25033f5-dc27-4af3-ad15-fc740bdc1657 None None] MFA Rules not processed for user `551281a31ce24e2f9b81b36f69419311`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:00.828306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e25033f5-dc27-4af3-ad15-fc740bdc1657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:00.973855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 683/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jul 28 03:52:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:05.154933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b24310-d2f1-49c1-9266-92ddb297dc9e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:05.155794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b24310-d2f1-49c1-9266-92ddb297dc9e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:05.156057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b24310-d2f1-49c1-9266-92ddb297dc9e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:05.157599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 669/1352] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jul 28 03:52:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:05.165214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e26f5f-afb3-465b-b7aa-49cc95d1fe22 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:05.165692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e26f5f-afb3-465b-b7aa-49cc95d1fe22 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:05.165905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e26f5f-afb3-465b-b7aa-49cc95d1fe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:05.222223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:05.253408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-59e26f5f-afb3-465b-b7aa-49cc95d1fe22 None None] MFA Rules not processed for user `551281a31ce24e2f9b81b36f69419311`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:05.254765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-59e26f5f-afb3-465b-b7aa-49cc95d1fe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:05.369557 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 684/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jul 28 03:52:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:05.411429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=59 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:06.732476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f302-2b6d-44fd-8605-bf07956cded8 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:06.733117 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f302-2b6d-44fd-8605-bf07956cded8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:06.733117 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f302-2b6d-44fd-8605-bf07956cded8 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:06.734378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 670/1354] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jul 28 03:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:06.739376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc9227-4fcb-4a8f-af62-225a7682579b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:06.739758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc9227-4fcb-4a8f-af62-225a7682579b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:06.740163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc9227-4fcb-4a8f-af62-225a7682579b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:06.818713 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fedc9227-4fcb-4a8f-af62-225a7682579b None None] MFA Rules not processed for user `551281a31ce24e2f9b81b36f69419311`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:06.819966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fedc9227-4fcb-4a8f-af62-225a7682579b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:06.861941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:06.927364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 685/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jul 28 03:52:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:10.389412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaa30b-922c-4cbc-a30d-d9b3742bcb7f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:10.389954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaa30b-922c-4cbc-a30d-d9b3742bcb7f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:10.389954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaa30b-922c-4cbc-a30d-d9b3742bcb7f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:10.395797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 671/1356] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:10.402542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dba78f-8b7d-4e76-b615-3817dcdc95e9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:10.405511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dba78f-8b7d-4e76-b615-3817dcdc95e9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:10.405821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dba78f-8b7d-4e76-b615-3817dcdc95e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:10.521515 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f0dba78f-8b7d-4e76-b615-3817dcdc95e9 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:10.522709 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f0dba78f-8b7d-4e76-b615-3817dcdc95e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:10.642401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 686/1357] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:10.699765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9bc12-053a-437d-87fd-abb58de2e10f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:10.700694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9bc12-053a-437d-87fd-abb58de2e10f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:10.701254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9bc12-053a-437d-87fd-abb58de2e10f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:10.703515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 672/1358] 10.222.0.46 () {54 vars in 962 bytes} [Thu Jul 28 03:52:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:10.710880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7356d1-1c43-4e4b-84e4-ed81642aa4b5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:10.711179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7356d1-1c43-4e4b-84e4-ed81642aa4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:10.711545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7356d1-1c43-4e4b-84e4-ed81642aa4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:10.735564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:10.801707 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fc7356d1-1c43-4e4b-84e4-ed81642aa4b5 None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:10.802977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fc7356d1-1c43-4e4b-84e4-ed81642aa4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:10.937602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 687/1359] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Jul 28 03:52:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:10.945871 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9bc12-053a-437d-87fd-abb58de2e10f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:10.947645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2ac9bc12-053a-437d-87fd-abb58de2e10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:11.101253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9bc12-053a-437d-87fd-abb58de2e10f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:11.187247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:11.228503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e79ce9-3730-4e46-8c2a-4e819b679968 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:11.229130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e79ce9-3730-4e46-8c2a-4e819b679968 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:11.229418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e79ce9-3730-4e46-8c2a-4e819b679968 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:11.229695 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e79ce9-3730-4e46-8c2a-4e819b679968 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:11.231438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a1e79ce9-3730-4e46-8c2a-4e819b679968 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:11.326509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e79ce9-3730-4e46-8c2a-4e819b679968 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:11.328164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e79ce9-3730-4e46-8c2a-4e819b679968 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:11.347543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 673/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:13.689683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c8a0f-c2ef-4e1d-aff1-e2977c0974da None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:13.690255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c8a0f-c2ef-4e1d-aff1-e2977c0974da None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:13.690255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c8a0f-c2ef-4e1d-aff1-e2977c0974da None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:13.691755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 688/1361] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:13.697788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb0fad-83cb-4462-9344-4e7fbca5af4f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:13.698265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb0fad-83cb-4462-9344-4e7fbca5af4f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:13.698368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb0fad-83cb-4462-9344-4e7fbca5af4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:13.781861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-22cb0fad-83cb-4462-9344-4e7fbca5af4f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:13.783117 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-22cb0fad-83cb-4462-9344-4e7fbca5af4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:13.842723 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:13.916173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 674/1362] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:13.960340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c8a0f-c2ef-4e1d-aff1-e2977c0974da None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:13.962625 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dc1c8a0f-c2ef-4e1d-aff1-e2977c0974da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:14.041383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:14.133447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c8a0f-c2ef-4e1d-aff1-e2977c0974da None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:14.261731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d3e7ad-d8bd-4538-92ec-d002aef86ceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:14.262616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3e7ad-d8bd-4538-92ec-d002aef86ceb service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:14.262925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3e7ad-d8bd-4538-92ec-d002aef86ceb service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:14.263221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3e7ad-d8bd-4538-92ec-d002aef86ceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:14.264863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-93d3e7ad-d8bd-4538-92ec-d002aef86ceb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:14.291115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:14.361836 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d3e7ad-d8bd-4538-92ec-d002aef86ceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:14.363541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d3e7ad-d8bd-4538-92ec-d002aef86ceb service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:14.383225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 689/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:16.728693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beffc52-431d-4466-a154-9525adf08c49 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:16.729684 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beffc52-431d-4466-a154-9525adf08c49 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:16.730194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beffc52-431d-4466-a154-9525adf08c49 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:16.731841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 675/1364] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:16.738452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812f23-be0a-4071-a955-ba34cb89adc9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:16.738992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812f23-be0a-4071-a955-ba34cb89adc9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:16.739719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812f23-be0a-4071-a955-ba34cb89adc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:16.815677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-19812f23-be0a-4071-a955-ba34cb89adc9 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:16.816897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-19812f23-be0a-4071-a955-ba34cb89adc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:16.920309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:16.926901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 690/1365] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:16.969294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a0eee-c9d5-46a4-9330-c4f625c5bc11 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:16.969655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a0eee-c9d5-46a4-9330-c4f625c5bc11 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:16.970007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a0eee-c9d5-46a4-9330-c4f625c5bc11 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:16.971472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 676/1366] 10.222.0.46 () {54 vars in 962 bytes} [Thu Jul 28 03:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:16.976768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723033f-02d9-44cf-acfb-3a655bfe8934 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:16.978072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723033f-02d9-44cf-acfb-3a655bfe8934 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:16.978369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723033f-02d9-44cf-acfb-3a655bfe8934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:17.058827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7723033f-02d9-44cf-acfb-3a655bfe8934 None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:17.059993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7723033f-02d9-44cf-acfb-3a655bfe8934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:17.185040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 691/1367] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Jul 28 03:52:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:17.193847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a0eee-c9d5-46a4-9330-c4f625c5bc11 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:17.195491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-752a0eee-c9d5-46a4-9330-c4f625c5bc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:17.362071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a0eee-c9d5-46a4-9330-c4f625c5bc11 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:17.384218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:17.482313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542767b-516b-4bfe-86b9-077a21c6e5bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:17.482882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542767b-516b-4bfe-86b9-077a21c6e5bb service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:17.483170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542767b-516b-4bfe-86b9-077a21c6e5bb service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:17.483446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542767b-516b-4bfe-86b9-077a21c6e5bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:17.485078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3542767b-516b-4bfe-86b9-077a21c6e5bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:17.587188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542767b-516b-4bfe-86b9-077a21c6e5bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:17.589427 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542767b-516b-4bfe-86b9-077a21c6e5bb service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:17.610637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 677/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:18.634342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277cd17-96fb-4ef4-8615-2d4fcd55f486 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:18.634991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277cd17-96fb-4ef4-8615-2d4fcd55f486 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:18.634991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277cd17-96fb-4ef4-8615-2d4fcd55f486 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:18.636458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 692/1369] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:18.641596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1320c-d525-4540-95c1-47e29288be1a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:18.642024 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1320c-d525-4540-95c1-47e29288be1a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:18.642208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1320c-d525-4540-95c1-47e29288be1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:18.675818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:18.715858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-92f1320c-d525-4540-95c1-47e29288be1a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:18.717074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-92f1320c-d525-4540-95c1-47e29288be1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:18.824668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 678/1370] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:19.183942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2667e7-2b8b-44dc-bc8b-fb6cd8880177 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:19.183942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2667e7-2b8b-44dc-bc8b-fb6cd8880177 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:19.183942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2667e7-2b8b-44dc-bc8b-fb6cd8880177 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:19.185908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 693/1371] 10.222.0.46 () {54 vars in 964 bytes} [Thu Jul 28 03:52:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:19.190197 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2838e-5070-4919-85cb-ca8091fe4042 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:19.190493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2838e-5070-4919-85cb-ca8091fe4042 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:19.190768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2838e-5070-4919-85cb-ca8091fe4042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:19.271078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d6b2838e-5070-4919-85cb-ca8091fe4042 None None] MFA Rules not processed for user `e2cf2558517445c79e47c3b67501ca09`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:19.272422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d6b2838e-5070-4919-85cb-ca8091fe4042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:19.323791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:19.357193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 679/1372] 10.222.0.46 () {58 vars in 1061 bytes} [Thu Jul 28 03:52:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:19.365036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2667e7-2b8b-44dc-bc8b-fb6cd8880177 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:19.366669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9d2667e7-2b8b-44dc-bc8b-fb6cd8880177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:19.423867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:19.484475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2667e7-2b8b-44dc-bc8b-fb6cd8880177 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:19.569135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e4da5-906b-46a8-92b4-e5dc67684171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:19.569762 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e4da5-906b-46a8-92b4-e5dc67684171 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:19.570043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e4da5-906b-46a8-92b4-e5dc67684171 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:19.570315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e4da5-906b-46a8-92b4-e5dc67684171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:19.572570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-269e4da5-906b-46a8-92b4-e5dc67684171 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:19.689751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:19.692298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee264398-5219-482a-b2f8-242e98d861e9 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:19.692943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee264398-5219-482a-b2f8-242e98d861e9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:19.692943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee264398-5219-482a-b2f8-242e98d861e9 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:19.694216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 680/1373] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:19.699422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78b528-5764-47d7-b58a-6f16363fb24f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:19.699604 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78b528-5764-47d7-b58a-6f16363fb24f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:19.699932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78b528-5764-47d7-b58a-6f16363fb24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:19.706301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e4da5-906b-46a8-92b4-e5dc67684171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:19.707816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e4da5-906b-46a8-92b4-e5dc67684171 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:19.728826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 694/1374] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:19.796582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cc78b528-5764-47d7-b58a-6f16363fb24f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:19.798969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cc78b528-5764-47d7-b58a-6f16363fb24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:19.974821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 681/1375] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:52:20.017150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e4da5-906b-46a8-92b4-e5dc67684171 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:20.018372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-269e4da5-906b-46a8-92b4-e5dc67684171 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:20.117293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e4da5-906b-46a8-92b4-e5dc67684171 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:20.220340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:20.233285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34cea96-0f6a-423d-9fbf-574d55d03758 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:20.233905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cea96-0f6a-423d-9fbf-574d55d03758 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:20.234192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cea96-0f6a-423d-9fbf-574d55d03758 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:20.234466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cea96-0f6a-423d-9fbf-574d55d03758 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:20.236243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d34cea96-0f6a-423d-9fbf-574d55d03758 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:20.331370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34cea96-0f6a-423d-9fbf-574d55d03758 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:20.333022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34cea96-0f6a-423d-9fbf-574d55d03758 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:20.351512 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 695/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:22.421128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3ac87-bdc5-4ccd-bf71-ef89cb059e0b None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:22.421624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3ac87-bdc5-4ccd-bf71-ef89cb059e0b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:22.421624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3ac87-bdc5-4ccd-bf71-ef89cb059e0b None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:22.422964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 682/1377] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:22.428156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c57e3d-ab59-45e8-aa24-bb9671f705d4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:22.428578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c57e3d-ab59-45e8-aa24-bb9671f705d4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:22.428798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c57e3d-ab59-45e8-aa24-bb9671f705d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:22.495904 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-59c57e3d-ab59-45e8-aa24-bb9671f705d4 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:22.497122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-59c57e3d-ab59-45e8-aa24-bb9671f705d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:22.556997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:22.605779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 696/1378] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:22.655517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3ac87-bdc5-4ccd-bf71-ef89cb059e0b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:22.657077 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-39e3ac87-bdc5-4ccd-bf71-ef89cb059e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:22.722566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:22.825574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3ac87-bdc5-4ccd-bf71-ef89cb059e0b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:22.936698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ee871-09c6-4cf9-b781-17de0269118a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:22.937312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ee871-09c6-4cf9-b781-17de0269118a service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:22.937597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ee871-09c6-4cf9-b781-17de0269118a service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:22.937882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ee871-09c6-4cf9-b781-17de0269118a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:22.939555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5e9ee871-09c6-4cf9-b781-17de0269118a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:22.961753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:23.034622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ee871-09c6-4cf9-b781-17de0269118a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:23.036545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ee871-09c6-4cf9-b781-17de0269118a service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:23.055206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 683/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:25.137117 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c0ca-e172-48a0-8d04-936df8c54204 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:25.137692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c0ca-e172-48a0-8d04-936df8c54204 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:25.137692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c0ca-e172-48a0-8d04-936df8c54204 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:25.139218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 697/1380] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:25.144430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6706405-09af-4f3b-b238-6781be81ed51 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:25.144879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6706405-09af-4f3b-b238-6781be81ed51 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:25.144986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6706405-09af-4f3b-b238-6781be81ed51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:25.216758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b6706405-09af-4f3b-b238-6781be81ed51 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:25.217963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b6706405-09af-4f3b-b238-6781be81ed51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:25.320359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:25.334878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 684/1381] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:25.380025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8c0ca-e172-48a0-8d04-936df8c54204 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:25.381821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9ff8c0ca-e172-48a0-8d04-936df8c54204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:25.543114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8c0ca-e172-48a0-8d04-936df8c54204 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:25.603505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:25.678941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb52efc-cd1f-4daa-93f2-64ce4880e6f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:25.680511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb52efc-cd1f-4daa-93f2-64ce4880e6f2 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:25.681023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb52efc-cd1f-4daa-93f2-64ce4880e6f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:25.681367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb52efc-cd1f-4daa-93f2-64ce4880e6f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:25.683041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6eb52efc-cd1f-4daa-93f2-64ce4880e6f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:25.781166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb52efc-cd1f-4daa-93f2-64ce4880e6f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:25.782859 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb52efc-cd1f-4daa-93f2-64ce4880e6f2 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:25.803305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 698/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:27.892330 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbaac4-a89b-4393-b4c2-6ff4cbf9e81e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:27.892892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbaac4-a89b-4393-b4c2-6ff4cbf9e81e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:27.892892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbaac4-a89b-4393-b4c2-6ff4cbf9e81e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:27.894220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 685/1383] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:27.900835 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbde64e-17e5-43e0-b591-c3182af17e44 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:27.901301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbde64e-17e5-43e0-b591-c3182af17e44 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:27.901569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbde64e-17e5-43e0-b591-c3182af17e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:27.973945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fdbde64e-17e5-43e0-b591-c3182af17e44 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:27.975583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fdbde64e-17e5-43e0-b591-c3182af17e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:27.978402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:28.088666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 699/1384] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:28.135859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbaac4-a89b-4393-b4c2-6ff4cbf9e81e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:28.137414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7acbaac4-a89b-4393-b4c2-6ff4cbf9e81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:28.312410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbaac4-a89b-4393-b4c2-6ff4cbf9e81e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:28.377208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:28.430004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f37b6-f449-4ea1-80de-ef9fc468a338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:28.430661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f37b6-f449-4ea1-80de-ef9fc468a338 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:28.430969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f37b6-f449-4ea1-80de-ef9fc468a338 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:28.431251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f37b6-f449-4ea1-80de-ef9fc468a338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:28.432921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2a1f37b6-f449-4ea1-80de-ef9fc468a338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:28.524882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f37b6-f449-4ea1-80de-ef9fc468a338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:28.526436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f37b6-f449-4ea1-80de-ef9fc468a338 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:28.546273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 686/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 03:52:30.585721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dc5a1-f7ae-4d0a-80be-96c46a418a09 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:30.587110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dc5a1-f7ae-4d0a-80be-96c46a418a09 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:30.587110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dc5a1-f7ae-4d0a-80be-96c46a418a09 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:30.588391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 700/1386] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:30.594216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41edab6-f378-4a17-b477-49096593ebd2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:30.594844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41edab6-f378-4a17-b477-49096593ebd2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:30.594971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41edab6-f378-4a17-b477-49096593ebd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:30.685080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f41edab6-f378-4a17-b477-49096593ebd2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:30.686673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f41edab6-f378-4a17-b477-49096593ebd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:30.718906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:30.794323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 687/1387] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:30.839185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dc5a1-f7ae-4d0a-80be-96c46a418a09 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:30.840718 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-671dc5a1-f7ae-4d0a-80be-96c46a418a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:30.986739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:31.015362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dc5a1-f7ae-4d0a-80be-96c46a418a09 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:31.155126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc9619-eafa-483b-9bce-5b4ca4bf965b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:31.155751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc9619-eafa-483b-9bce-5b4ca4bf965b service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:31.156040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc9619-eafa-483b-9bce-5b4ca4bf965b service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:31.156309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc9619-eafa-483b-9bce-5b4ca4bf965b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:31.157956 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-74fc9619-eafa-483b-9bce-5b4ca4bf965b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:31.255678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc9619-eafa-483b-9bce-5b4ca4bf965b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:31.257384 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc9619-eafa-483b-9bce-5b4ca4bf965b service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:31.263383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:31.278491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 701/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:33.327575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fd65-d224-4680-a5f0-9c2c76585176 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:33.328230 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fd65-d224-4680-a5f0-9c2c76585176 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:33.328230 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fd65-d224-4680-a5f0-9c2c76585176 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:33.329481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 688/1389] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:33.334935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf879-1716-41d5-9825-526337fe0f46 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:33.335376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf879-1716-41d5-9825-526337fe0f46 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:33.335484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf879-1716-41d5-9825-526337fe0f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:33.405077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d42cf879-1716-41d5-9825-526337fe0f46 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:33.406279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d42cf879-1716-41d5-9825-526337fe0f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:33.507404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 702/1390] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:33.552555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2fd65-d224-4680-a5f0-9c2c76585176 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:33.554088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2bc2fd65-d224-4680-a5f0-9c2c76585176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:33.701928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2fd65-d224-4680-a5f0-9c2c76585176 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:33.710757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:33.816214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c5248-c846-4a6d-a8e8-155d06295112 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:33.816824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c5248-c846-4a6d-a8e8-155d06295112 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:33.817108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c5248-c846-4a6d-a8e8-155d06295112 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:33.817373 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c5248-c846-4a6d-a8e8-155d06295112 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:33.819029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f23c5248-c846-4a6d-a8e8-155d06295112 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:33.907446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c5248-c846-4a6d-a8e8-155d06295112 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:33.909090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c5248-c846-4a6d-a8e8-155d06295112 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:33.930663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 689/1391] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:34.825294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12161627-9d8d-4c1f-a214-80afe901928e None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:34.825847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12161627-9d8d-4c1f-a214-80afe901928e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:34.826101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12161627-9d8d-4c1f-a214-80afe901928e None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:34.827520 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 703/1392] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:34.832804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4948b-5f3d-41dc-8945-17725ca15025 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:34.833202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4948b-5f3d-41dc-8945-17725ca15025 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:34.833510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4948b-5f3d-41dc-8945-17725ca15025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:34.849879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:34.901084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7cc4948b-5f3d-41dc-8945-17725ca15025 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:34.902296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7cc4948b-5f3d-41dc-8945-17725ca15025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:35.000485 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 690/1393] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:35.058114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6416a3-6b92-45fd-a7d6-c68ad9b49aeb None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:35.058441 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6416a3-6b92-45fd-a7d6-c68ad9b49aeb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:35.058678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6416a3-6b92-45fd-a7d6-c68ad9b49aeb None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:35.060099 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 704/1394] 10.222.0.46 () {54 vars in 966 bytes} [Thu Jul 28 03:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:35.065612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3173071-6ab0-4c5c-a0c0-c627707b379c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:35.065915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3173071-6ab0-4c5c-a0c0-c627707b379c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:35.066196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3173071-6ab0-4c5c-a0c0-c627707b379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:35.142501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e3173071-6ab0-4c5c-a0c0-c627707b379c None None] MFA Rules not processed for user `d8967641dd624dacb5ff12c8e22599bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:35.143653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e3173071-6ab0-4c5c-a0c0-c627707b379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:35.179200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:35.220667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 691/1395] 10.222.0.46 () {58 vars in 1063 bytes} [Thu Jul 28 03:52:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:35.228979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6416a3-6b92-45fd-a7d6-c68ad9b49aeb None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:35.230497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6d6416a3-6b92-45fd-a7d6-c68ad9b49aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:35.306527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:35.338867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6416a3-6b92-45fd-a7d6-c68ad9b49aeb None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:35.412344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d5e5d-8243-4b33-a7dd-d5eb445620bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8967641dd624dacb5ff12c8e22599bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:35.412884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d5e5d-8243-4b33-a7dd-d5eb445620bd service glance] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:35.413169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d5e5d-8243-4b33-a7dd-d5eb445620bd service glance] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:35.413448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d5e5d-8243-4b33-a7dd-d5eb445620bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:35.415702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-794d5e5d-8243-4b33-a7dd-d5eb445620bd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:35.526041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d5e5d-8243-4b33-a7dd-d5eb445620bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:35.527524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d5e5d-8243-4b33-a7dd-d5eb445620bd service glance] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:35.543853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:35.547405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 705/1396] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:35.951850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8ff2c-e260-4488-8a45-5cf3fb35009d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:35.952366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8ff2c-e260-4488-8a45-5cf3fb35009d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:35.952366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8ff2c-e260-4488-8a45-5cf3fb35009d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:35.953669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 692/1397] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:35.959193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78958d-c552-43c0-82fb-8a4f5776b969 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:35.959630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78958d-c552-43c0-82fb-8a4f5776b969 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:35.959743 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78958d-c552-43c0-82fb-8a4f5776b969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:36.027761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ae78958d-c552-43c0-82fb-8a4f5776b969 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:36.028947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ae78958d-c552-43c0-82fb-8a4f5776b969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:36.138632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 706/1398] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:36.186583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8ff2c-e260-4488-8a45-5cf3fb35009d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:36.188310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1de8ff2c-e260-4488-8a45-5cf3fb35009d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:36.292962 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6923-fe77-4705-bb5e-fdbf445fb51c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:36.293325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6923-fe77-4705-bb5e-fdbf445fb51c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:36.293671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6923-fe77-4705-bb5e-fdbf445fb51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:36.344298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8ff2c-e260-4488-8a45-5cf3fb35009d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:36.362147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9e7f6923-fe77-4705-bb5e-fdbf445fb51c None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:36.363424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9e7f6923-fe77-4705-bb5e-fdbf445fb51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:36.386997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:36.394402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:36.455894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 707/1399] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 03:52:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:52:36.462695 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc727cc-ab9f-41df-a710-8b299554a9d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:36.463366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc727cc-ab9f-41df-a710-8b299554a9d4 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:36.463645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc727cc-ab9f-41df-a710-8b299554a9d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:36.463918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc727cc-ab9f-41df-a710-8b299554a9d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:36.465910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2bc727cc-ab9f-41df-a710-8b299554a9d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:36.500152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a95c1f-feba-4c82-8843-33b379769d5e None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:36.500453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a95c1f-feba-4c82-8843-33b379769d5e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:36.500726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a95c1f-feba-4c82-8843-33b379769d5e None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:36.502091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 708/1400] 10.222.0.46 () {54 vars in 943 bytes} [Thu Jul 28 03:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:52:36.508422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf02fb-c409-4c54-a415-4499ddb592b1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:36.508714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf02fb-c409-4c54-a415-4499ddb592b1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:36.509048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf02fb-c409-4c54-a415-4499ddb592b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:36.574446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc727cc-ab9f-41df-a710-8b299554a9d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:36.576099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc727cc-ab9f-41df-a710-8b299554a9d4 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:36.594881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 693/1401] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 03:52:36.597750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-83cf02fb-c409-4c54-a415-4499ddb592b1 None None] MFA Rules not processed for user `19779569e08a42148d2933f644e3b9fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:36.599059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-83cf02fb-c409-4c54-a415-4499ddb592b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:36.719553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 709/1402] 10.222.0.46 () {58 vars in 1040 bytes} [Thu Jul 28 03:52:36 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:52:36.730185 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc727cc-ab9f-41df-a710-8b299554a9d4 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:36.731472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2bc727cc-ab9f-41df-a710-8b299554a9d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:36.781536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:36.825674 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc727cc-ab9f-41df-a710-8b299554a9d4 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:36.931821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2170b-6b02-4fc8-9e0b-8d0c971e1cdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:36.932735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2170b-6b02-4fc8-9e0b-8d0c971e1cdf service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:36.933016 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2170b-6b02-4fc8-9e0b-8d0c971e1cdf service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:36.933293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2170b-6b02-4fc8-9e0b-8d0c971e1cdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:36.934999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d1f2170b-6b02-4fc8-9e0b-8d0c971e1cdf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:37.016386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:37.020619 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2170b-6b02-4fc8-9e0b-8d0c971e1cdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:37.022228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2170b-6b02-4fc8-9e0b-8d0c971e1cdf service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:37.031399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 694/1403] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 03:52:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 03:52:38.666469 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427cd74-ae83-4aaa-abf5-4242c5031f33 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:38.667074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427cd74-ae83-4aaa-abf5-4242c5031f33 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:38.667074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427cd74-ae83-4aaa-abf5-4242c5031f33 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:38.669701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 710/1404] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:38.673684 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590e8f3-b988-43eb-8a29-36f62daea4d0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:38.674134 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590e8f3-b988-43eb-8a29-36f62daea4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:38.674287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590e8f3-b988-43eb-8a29-36f62daea4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:38.744605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f590e8f3-b988-43eb-8a29-36f62daea4d0 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:38.745862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f590e8f3-b988-43eb-8a29-36f62daea4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:38.850993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 695/1405] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:52:38.896103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427cd74-ae83-4aaa-abf5-4242c5031f33 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:38.897651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8427cd74-ae83-4aaa-abf5-4242c5031f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:38.901135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:39.067767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427cd74-ae83-4aaa-abf5-4242c5031f33 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:39.155267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:39.203300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405d08d-75bf-4cf1-9a22-c22461180386 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:39.203926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405d08d-75bf-4cf1-9a22-c22461180386 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:39.204214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405d08d-75bf-4cf1-9a22-c22461180386 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:39.204495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405d08d-75bf-4cf1-9a22-c22461180386 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:39.206144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0405d08d-75bf-4cf1-9a22-c22461180386 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:39.316302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405d08d-75bf-4cf1-9a22-c22461180386 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:39.317907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405d08d-75bf-4cf1-9a22-c22461180386 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:39.337271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 711/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:41.435188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46083840-34d8-4ab7-ab6f-0abbd4648ecb None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:41.435748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46083840-34d8-4ab7-ab6f-0abbd4648ecb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:41.435748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46083840-34d8-4ab7-ab6f-0abbd4648ecb None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:41.437071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 696/1407] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:41.442270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd860e-36f4-475d-b0de-b0fedf247bd7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:41.443192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd860e-36f4-475d-b0de-b0fedf247bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:41.443474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd860e-36f4-475d-b0de-b0fedf247bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:41.520178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bdcd860e-36f4-475d-b0de-b0fedf247bd7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:41.521724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bdcd860e-36f4-475d-b0de-b0fedf247bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:41.563837 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:41.629014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 712/1408] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:41.674693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46083840-34d8-4ab7-ab6f-0abbd4648ecb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:52:41.677944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-46083840-34d8-4ab7-ab6f-0abbd4648ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:41.758130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:41.837731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46083840-34d8-4ab7-ab6f-0abbd4648ecb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:52:41.968116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbbdcc-a6d5-4eb1-a953-63ced8ebf5ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:52:41.968858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbbdcc-a6d5-4eb1-a953-63ced8ebf5ae service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:41.969248 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbbdcc-a6d5-4eb1-a953-63ced8ebf5ae service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:41.969596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbbdcc-a6d5-4eb1-a953-63ced8ebf5ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:41.971336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-55cbbdcc-a6d5-4eb1-a953-63ced8ebf5ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:42.012211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:42.074455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cbbdcc-a6d5-4eb1-a953-63ced8ebf5ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:52:42.076105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cbbdcc-a6d5-4eb1-a953-63ced8ebf5ae service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:52:42.095672 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 697/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:52:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:52:51.626601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=26 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:52.092968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=30 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:59.278812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6b7c9-9276-4edd-929e-3853a0514cc1 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:59.279377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6b7c9-9276-4edd-929e-3853a0514cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:59.279377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6b7c9-9276-4edd-929e-3853a0514cc1 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:59.280741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 713/1410] 10.222.0.46 () {54 vars in 918 bytes} [Thu Jul 28 03:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:59.286015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911fbc3-0299-43fa-820c-1ac4c7825a17 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:59.286504 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911fbc3-0299-43fa-820c-1ac4c7825a17 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:59.286921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911fbc3-0299-43fa-820c-1ac4c7825a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:59.363081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4911fbc3-0299-43fa-820c-1ac4c7825a17 None None] MFA Rules not processed for user `551281a31ce24e2f9b81b36f69419311`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:59.364297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4911fbc3-0299-43fa-820c-1ac4c7825a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:59.470567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 698/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Jul 28 03:52:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:52:59.485112 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde750e5-4d95-438d-8c04-f37e990e5640 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:59.485398 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde750e5-4d95-438d-8c04-f37e990e5640 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:59.485677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde750e5-4d95-438d-8c04-f37e990e5640 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:59.487060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 714/1412] 10.222.0.46 () {54 vars in 918 bytes} [Thu Jul 28 03:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:52:59.491542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388aa5-4308-40eb-bac2-90d955c3cce7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:52:59.492710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388aa5-4308-40eb-bac2-90d955c3cce7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:52:59.493075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388aa5-4308-40eb-bac2-90d955c3cce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:52:59.566662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:52:59.569407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-56388aa5-4308-40eb-bac2-90d955c3cce7 None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:52:59.571280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-56388aa5-4308-40eb-bac2-90d955c3cce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:52:59.686626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 699/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Jul 28 03:52:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:02.191494 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b45c5-f175-48c2-9e49-d16417425da9 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:02.192017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b45c5-f175-48c2-9e49-d16417425da9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:02.192017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b45c5-f175-48c2-9e49-d16417425da9 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:02.193351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:02.199500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa486e-2f40-42b1-9ce0-c5924729b909 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:02.199899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa486e-2f40-42b1-9ce0-c5924729b909 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:02.200064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa486e-2f40-42b1-9ce0-c5924729b909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:02.265936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-efaa486e-2f40-42b1-9ce0-c5924729b909 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:02.267183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-efaa486e-2f40-42b1-9ce0-c5924729b909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:02.324938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:02.367176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:02.411084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b45c5-f175-48c2-9e49-d16417425da9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:02.412868 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-eb2b45c5-f175-48c2-9e49-d16417425da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:02.557039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b45c5-f175-48c2-9e49-d16417425da9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:02.627824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:02.667390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d1a5b-528d-4af2-b02d-36df323376e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:02.667998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d1a5b-528d-4af2-b02d-36df323376e6 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:02.668277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d1a5b-528d-4af2-b02d-36df323376e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:02.668548 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d1a5b-528d-4af2-b02d-36df323376e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:02.670161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-405d1a5b-528d-4af2-b02d-36df323376e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:02.760359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d1a5b-528d-4af2-b02d-36df323376e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:02.761934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d1a5b-528d-4af2-b02d-36df323376e6 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:02.779906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 716/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:03.681883 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0919bc4-6ed5-4730-87b9-421b6d5811c3 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:03.682436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0919bc4-6ed5-4730-87b9-421b6d5811c3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:03.682436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0919bc4-6ed5-4730-87b9-421b6d5811c3 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:03.683828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:03.688693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e314f-7bcd-4efd-8cb2-13f5b5dc154c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:03.689176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e314f-7bcd-4efd-8cb2-13f5b5dc154c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:03.689274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e314f-7bcd-4efd-8cb2-13f5b5dc154c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:03.766147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-dc7e314f-7bcd-4efd-8cb2-13f5b5dc154c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:03.767391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dc7e314f-7bcd-4efd-8cb2-13f5b5dc154c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:03.795139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:03.863482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:04.013442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0919bc4-6ed5-4730-87b9-421b6d5811c3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:04.014993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a0919bc4-6ed5-4730-87b9-421b6d5811c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:04.160608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0919bc4-6ed5-4730-87b9-421b6d5811c3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:04.201008 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:04.274017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46cad8-14d7-40ec-9ef2-4cbb8e361d12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:04.274631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46cad8-14d7-40ec-9ef2-4cbb8e361d12 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:04.274942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46cad8-14d7-40ec-9ef2-4cbb8e361d12 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:04.275216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46cad8-14d7-40ec-9ef2-4cbb8e361d12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:04.276854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7b46cad8-14d7-40ec-9ef2-4cbb8e361d12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:04.363243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46cad8-14d7-40ec-9ef2-4cbb8e361d12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:04.364748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46cad8-14d7-40ec-9ef2-4cbb8e361d12 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:04.383470 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 702/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:05.291450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09788-e7da-415d-8b5d-e5e6bbcaf03b None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:05.292028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09788-e7da-415d-8b5d-e5e6bbcaf03b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:05.292028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09788-e7da-415d-8b5d-e5e6bbcaf03b None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:05.293359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:05.298301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e172d1-43ae-4cff-aadb-a60f8b314595 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:05.298743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e172d1-43ae-4cff-aadb-a60f8b314595 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:05.298877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e172d1-43ae-4cff-aadb-a60f8b314595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:05.353051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:05.375653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-86e172d1-43ae-4cff-aadb-a60f8b314595 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:05.376872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-86e172d1-43ae-4cff-aadb-a60f8b314595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:05.473059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:53:05.630177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e09788-e7da-415d-8b5d-e5e6bbcaf03b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:05.631726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a6e09788-e7da-415d-8b5d-e5e6bbcaf03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:05.774436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e09788-e7da-415d-8b5d-e5e6bbcaf03b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:05.786936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:05.888308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969da134-e678-4b97-aec3-ba747cd1cd7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:05.888870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969da134-e678-4b97-aec3-ba747cd1cd7d service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:05.889157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969da134-e678-4b97-aec3-ba747cd1cd7d service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:05.889435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969da134-e678-4b97-aec3-ba747cd1cd7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:05.891207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-969da134-e678-4b97-aec3-ba747cd1cd7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:05.980502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969da134-e678-4b97-aec3-ba747cd1cd7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:05.982105 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969da134-e678-4b97-aec3-ba747cd1cd7d service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:05.999993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 719/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:06.874671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba62bf-88de-42b5-b100-d885ccc1d92f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:06.875263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba62bf-88de-42b5-b100-d885ccc1d92f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:06.875376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba62bf-88de-42b5-b100-d885ccc1d92f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:06.876655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:06.881658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efa91f-9461-4f49-b078-30a43df65b3d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:06.882090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efa91f-9461-4f49-b078-30a43df65b3d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:06.882956 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efa91f-9461-4f49-b078-30a43df65b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:06.903096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:06.967176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-53efa91f-9461-4f49-b078-30a43df65b3d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:06.968374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-53efa91f-9461-4f49-b078-30a43df65b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:07.063522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:07.213259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba62bf-88de-42b5-b100-d885ccc1d92f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:07.215003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b2ba62bf-88de-42b5-b100-d885ccc1d92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:07.327557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:07.364627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba62bf-88de-42b5-b100-d885ccc1d92f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:07.478217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c476f4d-5e71-46a7-ac34-a49048fde403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:07.478750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c476f4d-5e71-46a7-ac34-a49048fde403 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:07.479067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c476f4d-5e71-46a7-ac34-a49048fde403 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:07.479340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c476f4d-5e71-46a7-ac34-a49048fde403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:07.480960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3c476f4d-5e71-46a7-ac34-a49048fde403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:07.560711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:07.575211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c476f4d-5e71-46a7-ac34-a49048fde403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:07.576837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c476f4d-5e71-46a7-ac34-a49048fde403 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:07.595600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 705/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:08.471756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583b775-382b-4c0b-88eb-054bdddc7334 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:08.472284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583b775-382b-4c0b-88eb-054bdddc7334 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:08.472284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583b775-382b-4c0b-88eb-054bdddc7334 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:08.473600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:08.479197 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4d14c-e67c-4ef8-aaf7-760fa6e5b181 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:08.479658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4d14c-e67c-4ef8-aaf7-760fa6e5b181 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:08.479771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4d14c-e67c-4ef8-aaf7-760fa6e5b181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:08.555799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6ea4d14c-e67c-4ef8-aaf7-760fa6e5b181 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:08.556997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6ea4d14c-e67c-4ef8-aaf7-760fa6e5b181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:08.654297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:08.805426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583b775-382b-4c0b-88eb-054bdddc7334 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:08.807017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1583b775-382b-4c0b-88eb-054bdddc7334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:08.891358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:08.955224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583b775-382b-4c0b-88eb-054bdddc7334 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:09.066872 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a0340-7071-4004-aae3-9ac2c7efd64c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:09.067431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0340-7071-4004-aae3-9ac2c7efd64c service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:09.067713 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0340-7071-4004-aae3-9ac2c7efd64c service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:09.067985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0340-7071-4004-aae3-9ac2c7efd64c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:09.069671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bf8a0340-7071-4004-aae3-9ac2c7efd64c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:09.116649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:09.157938 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a0340-7071-4004-aae3-9ac2c7efd64c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:09.159575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a0340-7071-4004-aae3-9ac2c7efd64c service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:09.178023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 722/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:09.985616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482cb44-bd2a-472a-b968-0dd7ebb2d489 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:09.986163 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482cb44-bd2a-472a-b968-0dd7ebb2d489 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:09.986163 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482cb44-bd2a-472a-b968-0dd7ebb2d489 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:09.987561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:09.993159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e29e3-4adf-447f-83bc-54e4639bb49f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:09.993609 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e29e3-4adf-447f-83bc-54e4639bb49f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:09.993727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e29e3-4adf-447f-83bc-54e4639bb49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:10.069767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d33e29e3-4adf-447f-83bc-54e4639bb49f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:10.071014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d33e29e3-4adf-447f-83bc-54e4639bb49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:10.167725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:10.322013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482cb44-bd2a-472a-b968-0dd7ebb2d489 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:10.323590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6482cb44-bd2a-472a-b968-0dd7ebb2d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:10.372465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:10.469731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482cb44-bd2a-472a-b968-0dd7ebb2d489 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:10.586224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d5796-b9c6-41b5-a78c-bf00b99bd6bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:10.586752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d5796-b9c6-41b5-a78c-bf00b99bd6bd service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:10.587095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d5796-b9c6-41b5-a78c-bf00b99bd6bd service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:10.587367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d5796-b9c6-41b5-a78c-bf00b99bd6bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:10.589001 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d57d5796-b9c6-41b5-a78c-bf00b99bd6bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:10.596617 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:10.683335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d5796-b9c6-41b5-a78c-bf00b99bd6bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:10.684959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d5796-b9c6-41b5-a78c-bf00b99bd6bd service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:10.703727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 708/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:11.508131 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb880c5-4855-4acd-bb03-c57eddca45f2 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:11.508614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb880c5-4855-4acd-bb03-c57eddca45f2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:11.508726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb880c5-4855-4acd-bb03-c57eddca45f2 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:11.510035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:53:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:11.514907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61031269-a513-44a2-a970-50784c7aee0e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:11.515313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61031269-a513-44a2-a970-50784c7aee0e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:11.515468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61031269-a513-44a2-a970-50784c7aee0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:11.594350 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-61031269-a513-44a2-a970-50784c7aee0e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:11.595594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-61031269-a513-44a2-a970-50784c7aee0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:11.661352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:11.690979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:11.841116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb880c5-4855-4acd-bb03-c57eddca45f2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:11.842592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0cb880c5-4855-4acd-bb03-c57eddca45f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:11.860922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:11.989286 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb880c5-4855-4acd-bb03-c57eddca45f2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:12.074894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:12.101917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c4917-51a8-4956-ba65-906a71200fba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:12.102471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c4917-51a8-4956-ba65-906a71200fba service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:12.102754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c4917-51a8-4956-ba65-906a71200fba service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:12.103061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c4917-51a8-4956-ba65-906a71200fba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:12.104692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5a1c4917-51a8-4956-ba65-906a71200fba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:12.192825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c4917-51a8-4956-ba65-906a71200fba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:12.194477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c4917-51a8-4956-ba65-906a71200fba service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:12.212769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 725/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:13.028043 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec0a17-3d97-4ee8-a7e1-1c696a3484fc None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:13.028534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec0a17-3d97-4ee8-a7e1-1c696a3484fc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:13.028672 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec0a17-3d97-4ee8-a7e1-1c696a3484fc None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:13.029982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:13.035042 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d36a5-bcd4-421a-ba9c-ca8a6cbaeba3 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:13.035424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d36a5-bcd4-421a-ba9c-ca8a6cbaeba3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:13.035638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d36a5-bcd4-421a-ba9c-ca8a6cbaeba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:13.113030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-124d36a5-bcd4-421a-ba9c-ca8a6cbaeba3 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:13.114266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-124d36a5-bcd4-421a-ba9c-ca8a6cbaeba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:13.156945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:13.219552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:13.371613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec0a17-3d97-4ee8-a7e1-1c696a3484fc None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:13.373172 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-23ec0a17-3d97-4ee8-a7e1-1c696a3484fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:13.515459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec0a17-3d97-4ee8-a7e1-1c696a3484fc None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:13.569311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:13.627671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb2519-965d-4f13-9e48-0e8d2dd65c97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:13.628214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb2519-965d-4f13-9e48-0e8d2dd65c97 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:13.628494 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb2519-965d-4f13-9e48-0e8d2dd65c97 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:13.628766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb2519-965d-4f13-9e48-0e8d2dd65c97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:13.630385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-37cb2519-965d-4f13-9e48-0e8d2dd65c97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:13.717313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb2519-965d-4f13-9e48-0e8d2dd65c97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:13.718928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb2519-965d-4f13-9e48-0e8d2dd65c97 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:13.736251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 711/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:14.542644 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c01d7-1307-452d-b89a-c494950fe978 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:14.543168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c01d7-1307-452d-b89a-c494950fe978 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:14.543267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c01d7-1307-452d-b89a-c494950fe978 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:14.544593 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:14.549445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74ef57-bfef-4653-b9c4-5b7a4bf3149c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:14.549857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74ef57-bfef-4653-b9c4-5b7a4bf3149c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:14.550005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74ef57-bfef-4653-b9c4-5b7a4bf3149c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:14.626079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-df74ef57-bfef-4653-b9c4-5b7a4bf3149c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:14.627323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-df74ef57-bfef-4653-b9c4-5b7a4bf3149c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:14.629338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:14.723262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:53:14.874249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c01d7-1307-452d-b89a-c494950fe978 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:14.875840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-641c01d7-1307-452d-b89a-c494950fe978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:15.023459 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c01d7-1307-452d-b89a-c494950fe978 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:15.050883 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:15.155210 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29f454-4bf3-470e-9b0f-5e7e97dcb0d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:15.155681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29f454-4bf3-470e-9b0f-5e7e97dcb0d9 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:15.155964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29f454-4bf3-470e-9b0f-5e7e97dcb0d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:15.156240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29f454-4bf3-470e-9b0f-5e7e97dcb0d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:15.157888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9d29f454-4bf3-470e-9b0f-5e7e97dcb0d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:15.275491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29f454-4bf3-470e-9b0f-5e7e97dcb0d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:15.277758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29f454-4bf3-470e-9b0f-5e7e97dcb0d9 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:15.297467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 728/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:17.310442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d68a29-e365-4ec4-b3b6-f69c88c499a7 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:17.311022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d68a29-e365-4ec4-b3b6-f69c88c499a7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:17.311022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d68a29-e365-4ec4-b3b6-f69c88c499a7 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:17.312581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:17.317648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e3e92-0f00-44ab-af26-99f76df12346 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:17.318041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e3e92-0f00-44ab-af26-99f76df12346 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:17.318199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e3e92-0f00-44ab-af26-99f76df12346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:17.360013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:17.385225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-966e3e92-0f00-44ab-af26-99f76df12346 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:17.386422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-966e3e92-0f00-44ab-af26-99f76df12346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:17.481096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:17.520321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d68a29-e365-4ec4-b3b6-f69c88c499a7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:17.521849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-28d68a29-e365-4ec4-b3b6-f69c88c499a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:17.623797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d68a29-e365-4ec4-b3b6-f69c88c499a7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:17.655910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:17.696377 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243d093-48a2-4810-b57f-c3fe255767ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:17.696982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243d093-48a2-4810-b57f-c3fe255767ba service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:17.697263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243d093-48a2-4810-b57f-c3fe255767ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:17.697534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243d093-48a2-4810-b57f-c3fe255767ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:17.699176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a243d093-48a2-4810-b57f-c3fe255767ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:17.805073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243d093-48a2-4810-b57f-c3fe255767ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:17.806492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243d093-48a2-4810-b57f-c3fe255767ba service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:17.824721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 714/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:20.088657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a062c0-e7f3-4d6e-900a-e94ce231f4f2 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:20.089158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a062c0-e7f3-4d6e-900a-e94ce231f4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:20.089262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a062c0-e7f3-4d6e-900a-e94ce231f4f2 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:20.090598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:20.095731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f374e-ed79-4c2c-94cb-21003e973c03 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:20.096140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f374e-ed79-4c2c-94cb-21003e973c03 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:20.096287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f374e-ed79-4c2c-94cb-21003e973c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:20.162361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2c3f374e-ed79-4c2c-94cb-21003e973c03 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:20.163763 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2c3f374e-ed79-4c2c-94cb-21003e973c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:20.176238 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:20.260923 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:20.298514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a062c0-e7f3-4d6e-900a-e94ce231f4f2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:20.300124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-60a062c0-e7f3-4d6e-900a-e94ce231f4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:20.403288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a062c0-e7f3-4d6e-900a-e94ce231f4f2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:20.415609 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:20.475987 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37db160-44d9-46a5-8545-92038e8db2a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:20.476594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37db160-44d9-46a5-8545-92038e8db2a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:20.476874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37db160-44d9-46a5-8545-92038e8db2a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:20.477145 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37db160-44d9-46a5-8545-92038e8db2a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:20.478759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e37db160-44d9-46a5-8545-92038e8db2a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:20.588749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37db160-44d9-46a5-8545-92038e8db2a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:20.590187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37db160-44d9-46a5-8545-92038e8db2a7 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:20.608459 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 731/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:22.957817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a618d-7f9e-42bb-9db7-d4c0878fc220 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:22.958385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a618d-7f9e-42bb-9db7-d4c0878fc220 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:22.958385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a618d-7f9e-42bb-9db7-d4c0878fc220 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:22.959821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:22.965694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28041905-607f-4f7c-84b1-159b60786af4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:22.966108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28041905-607f-4f7c-84b1-159b60786af4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:22.966255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28041905-607f-4f7c-84b1-159b60786af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:23.020088 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:23.039176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-28041905-607f-4f7c-84b1-159b60786af4 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:23.040392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-28041905-607f-4f7c-84b1-159b60786af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:23.255611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:53:23.295593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a618d-7f9e-42bb-9db7-d4c0878fc220 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:23.297130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-672a618d-7f9e-42bb-9db7-d4c0878fc220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:23.519397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a618d-7f9e-42bb-9db7-d4c0878fc220 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:23.560620 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:23.594093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea1c7d-21e6-4205-a5cf-6ec088436188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:23.594700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1c7d-21e6-4205-a5cf-6ec088436188 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:23.595022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1c7d-21e6-4205-a5cf-6ec088436188 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:23.595295 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1c7d-21e6-4205-a5cf-6ec088436188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:23.596910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-90ea1c7d-21e6-4205-a5cf-6ec088436188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:23.704110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea1c7d-21e6-4205-a5cf-6ec088436188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:23.705534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea1c7d-21e6-4205-a5cf-6ec088436188 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:23.723457 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 717/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 03:53:25.836531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe805e-2b7d-4176-aab7-2b19a86f6399 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:25.837068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe805e-2b7d-4176-aab7-2b19a86f6399 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:25.837068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe805e-2b7d-4176-aab7-2b19a86f6399 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:25.838449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:25.843616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf980c3-7460-4d96-9796-26b519269ce8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:25.844015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf980c3-7460-4d96-9796-26b519269ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:25.844173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf980c3-7460-4d96-9796-26b519269ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:25.909989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9cf980c3-7460-4d96-9796-26b519269ce8 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:25.911227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9cf980c3-7460-4d96-9796-26b519269ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:25.932687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:26.007663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:26.046561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe805e-2b7d-4176-aab7-2b19a86f6399 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:26.048104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-94fe805e-2b7d-4176-aab7-2b19a86f6399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:26.151621 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe805e-2b7d-4176-aab7-2b19a86f6399 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:26.173334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:26.226808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d373d-6dc9-4f8c-b58a-512b8c811366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:26.227434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d373d-6dc9-4f8c-b58a-512b8c811366 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:26.227715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d373d-6dc9-4f8c-b58a-512b8c811366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:26.227985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d373d-6dc9-4f8c-b58a-512b8c811366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:26.229590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-857d373d-6dc9-4f8c-b58a-512b8c811366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:26.339559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d373d-6dc9-4f8c-b58a-512b8c811366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:26.340987 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d373d-6dc9-4f8c-b58a-512b8c811366 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:26.358725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 734/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:28.416412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd38cb-357e-40df-a3e0-ffc6965a212d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:28.416944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd38cb-357e-40df-a3e0-ffc6965a212d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:28.416944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd38cb-357e-40df-a3e0-ffc6965a212d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:28.418230 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:28.423546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff88c3-5068-492f-b0e5-80a193c5ea45 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:28.423977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff88c3-5068-492f-b0e5-80a193c5ea45 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:28.424108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff88c3-5068-492f-b0e5-80a193c5ea45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:28.486992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:28.489705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ffff88c3-5068-492f-b0e5-80a193c5ea45 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:28.492063 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ffff88c3-5068-492f-b0e5-80a193c5ea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:28.596239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:28.635749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd38cb-357e-40df-a3e0-ffc6965a212d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:28.637283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-01fd38cb-357e-40df-a3e0-ffc6965a212d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:28.740816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd38cb-357e-40df-a3e0-ffc6965a212d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:28.789861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:28.814701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea5efe-db4a-4417-ae5c-e7faf0dd21d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:28.815339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea5efe-db4a-4417-ae5c-e7faf0dd21d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:28.815624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea5efe-db4a-4417-ae5c-e7faf0dd21d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:28.815900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea5efe-db4a-4417-ae5c-e7faf0dd21d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:28.817516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-22ea5efe-db4a-4417-ae5c-e7faf0dd21d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:28.925317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea5efe-db4a-4417-ae5c-e7faf0dd21d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:28.926760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea5efe-db4a-4417-ae5c-e7faf0dd21d3 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:28.945057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 720/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:31.021622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883da2-fb1d-463f-a2c0-e939109cddb1 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:31.022504 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883da2-fb1d-463f-a2c0-e939109cddb1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:31.022504 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883da2-fb1d-463f-a2c0-e939109cddb1 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:31.023717 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:31.029135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89b45a-0a40-4d5d-88bb-ce770656ded5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:31.029577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89b45a-0a40-4d5d-88bb-ce770656ded5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:31.029694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89b45a-0a40-4d5d-88bb-ce770656ded5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:31.095279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0c89b45a-0a40-4d5d-88bb-ce770656ded5 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:31.096488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0c89b45a-0a40-4d5d-88bb-ce770656ded5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:31.128587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:31.196440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:31.234463 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c883da2-fb1d-463f-a2c0-e939109cddb1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:31.236041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6c883da2-fb1d-463f-a2c0-e939109cddb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:31.339678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c883da2-fb1d-463f-a2c0-e939109cddb1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:31.368492 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:31.415375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce885c53-d3f5-4d11-96dd-332f675d0330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:31.415991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce885c53-d3f5-4d11-96dd-332f675d0330 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:31.416268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce885c53-d3f5-4d11-96dd-332f675d0330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:31.416536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce885c53-d3f5-4d11-96dd-332f675d0330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:31.418179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ce885c53-d3f5-4d11-96dd-332f675d0330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:31.526016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce885c53-d3f5-4d11-96dd-332f675d0330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:31.527546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce885c53-d3f5-4d11-96dd-332f675d0330 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:31.546541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 737/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:33.598375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4252df3-36a7-434a-b009-7410f239628a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:33.598939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4252df3-36a7-434a-b009-7410f239628a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:33.599128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4252df3-36a7-434a-b009-7410f239628a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:33.600899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:33.606234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01057cfe-a9be-48ea-809c-bad39569211b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:33.606648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01057cfe-a9be-48ea-809c-bad39569211b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:33.606802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01057cfe-a9be-48ea-809c-bad39569211b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:33.671699 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-01057cfe-a9be-48ea-809c-bad39569211b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:33.672899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-01057cfe-a9be-48ea-809c-bad39569211b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:33.681463 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:33.768113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:53:33.806777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4252df3-36a7-434a-b009-7410f239628a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:33.808318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c4252df3-36a7-434a-b009-7410f239628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:33.910658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4252df3-36a7-434a-b009-7410f239628a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:33.966877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:33.984945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57cda3-ee84-4b77-83ed-64453bef9916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:33.985549 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57cda3-ee84-4b77-83ed-64453bef9916 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:33.985825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57cda3-ee84-4b77-83ed-64453bef9916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:33.986094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57cda3-ee84-4b77-83ed-64453bef9916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:33.987739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2d57cda3-ee84-4b77-83ed-64453bef9916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:34.093498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57cda3-ee84-4b77-83ed-64453bef9916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:34.094949 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57cda3-ee84-4b77-83ed-64453bef9916 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:34.112743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 723/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:36.153489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8180b8-a150-4d3f-b31a-e111cf225c60 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:36.154014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8180b8-a150-4d3f-b31a-e111cf225c60 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:36.154366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8180b8-a150-4d3f-b31a-e111cf225c60 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:36.155796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:36.161118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b12bd2-d349-45cd-b81f-44cf98a92167 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:36.161558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b12bd2-d349-45cd-b81f-44cf98a92167 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:36.161664 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b12bd2-d349-45cd-b81f-44cf98a92167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:36.227406 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-36b12bd2-d349-45cd-b81f-44cf98a92167 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:36.228590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-36b12bd2-d349-45cd-b81f-44cf98a92167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:36.267425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:36.326438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:53:36.364450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8180b8-a150-4d3f-b31a-e111cf225c60 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:36.366658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-de8180b8-a150-4d3f-b31a-e111cf225c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:36.469793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8180b8-a150-4d3f-b31a-e111cf225c60 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:36.508116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:36.547465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e054749-a9ce-4a6d-ae5b-03d43503b4ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:36.548130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e054749-a9ce-4a6d-ae5b-03d43503b4ec service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:36.548460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e054749-a9ce-4a6d-ae5b-03d43503b4ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:36.548735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e054749-a9ce-4a6d-ae5b-03d43503b4ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:36.550477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0e054749-a9ce-4a6d-ae5b-03d43503b4ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:36.658199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e054749-a9ce-4a6d-ae5b-03d43503b4ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:36.659599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e054749-a9ce-4a6d-ae5b-03d43503b4ec service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:36.678029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 740/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:38.723177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008671e-3c7a-43f9-851b-d182b3377584 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:38.723742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008671e-3c7a-43f9-851b-d182b3377584 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:38.723742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008671e-3c7a-43f9-851b-d182b3377584 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:38.725687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:38.731138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81880d3-a766-4f3a-aa9f-de32aa8322fb None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:38.731570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81880d3-a766-4f3a-aa9f-de32aa8322fb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:38.731706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81880d3-a766-4f3a-aa9f-de32aa8322fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:38.797400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c81880d3-a766-4f3a-aa9f-de32aa8322fb None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:38.798615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c81880d3-a766-4f3a-aa9f-de32aa8322fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:38.816290 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:38.893831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:38.931207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008671e-3c7a-43f9-851b-d182b3377584 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:38.933319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7008671e-3c7a-43f9-851b-d182b3377584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:39.034348 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008671e-3c7a-43f9-851b-d182b3377584 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:39.102467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:39.109529 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88d58e-887b-44fa-bb5d-d741d2c17a10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:39.110155 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d58e-887b-44fa-bb5d-d741d2c17a10 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:39.110438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d58e-887b-44fa-bb5d-d741d2c17a10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:39.110710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d58e-887b-44fa-bb5d-d741d2c17a10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:39.112404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3c88d58e-887b-44fa-bb5d-d741d2c17a10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:39.221112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88d58e-887b-44fa-bb5d-d741d2c17a10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:39.222536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88d58e-887b-44fa-bb5d-d741d2c17a10 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:39.245552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 726/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 03:53:41.257407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c9f1b-5b4c-437a-aa4f-194af5871142 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:41.257991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c9f1b-5b4c-437a-aa4f-194af5871142 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:41.257991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c9f1b-5b4c-437a-aa4f-194af5871142 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:41.259354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:53:41.264612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6f0a1-c80c-4f94-a80e-702eded99f16 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:41.265054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6f0a1-c80c-4f94-a80e-702eded99f16 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:41.265158 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6f0a1-c80c-4f94-a80e-702eded99f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:41.330003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5ed6f0a1-c80c-4f94-a80e-702eded99f16 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:53:41.331231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5ed6f0a1-c80c-4f94-a80e-702eded99f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:41.379412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:41.427110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:53:41.465147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c9f1b-5b4c-437a-aa4f-194af5871142 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:53:41.466725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-960c9f1b-5b4c-437a-aa4f-194af5871142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:41.571214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c9f1b-5b4c-437a-aa4f-194af5871142 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:53:41.617386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:41.643480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157137c-8c40-42c6-a0ea-d2611ff73aa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:53:41.644093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157137c-8c40-42c6-a0ea-d2611ff73aa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:53:41.644374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157137c-8c40-42c6-a0ea-d2611ff73aa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:53:41.644648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157137c-8c40-42c6-a0ea-d2611ff73aa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:53:41.646272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e157137c-8c40-42c6-a0ea-d2611ff73aa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:53:41.752688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157137c-8c40-42c6-a0ea-d2611ff73aa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:53:41.754130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157137c-8c40-42c6-a0ea-d2611ff73aa3 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:53:41.772984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 743/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:53:51.425290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=22 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:53:51.769979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=66 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:01.314868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9c793-c3ad-42a7-82b3-b1fbc98f44b0 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:01.315426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9c793-c3ad-42a7-82b3-b1fbc98f44b0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:01.315426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9c793-c3ad-42a7-82b3-b1fbc98f44b0 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:01.316767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 728/1471] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:54:01.322858 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a507c-5d78-4e65-88c4-72cdeff69189 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:01.323354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a507c-5d78-4e65-88c4-72cdeff69189 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:01.323354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a507c-5d78-4e65-88c4-72cdeff69189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:01.389067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-759a507c-5d78-4e65-88c4-72cdeff69189 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:54:01.390274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-759a507c-5d78-4e65-88c4-72cdeff69189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:01.485677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 744/1472] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:54:01.571593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91a00f-4ca1-4970-8f5a-206f1dc27238 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:01.571879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91a00f-4ca1-4970-8f5a-206f1dc27238 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:01.572151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91a00f-4ca1-4970-8f5a-206f1dc27238 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:01.573492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 729/1473] 10.222.0.46 () {54 vars in 966 bytes} [Thu Jul 28 03:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:54:01.578610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bf2b0-ec32-465b-b304-e36c11b90235 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:01.578921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bf2b0-ec32-465b-b304-e36c11b90235 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:01.579199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bf2b0-ec32-465b-b304-e36c11b90235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:01.653453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-800bf2b0-ec32-465b-b304-e36c11b90235 None None] MFA Rules not processed for user `d8967641dd624dacb5ff12c8e22599bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:54:01.654747 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-800bf2b0-ec32-465b-b304-e36c11b90235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:01.674731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:01.731734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 745/1474] 10.222.0.46 () {58 vars in 1063 bytes} [Thu Jul 28 03:54:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:54:01.739378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91a00f-4ca1-4970-8f5a-206f1dc27238 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:54:01.740945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fe91a00f-4ca1-4970-8f5a-206f1dc27238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:01.843349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91a00f-4ca1-4970-8f5a-206f1dc27238 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:54:01.915151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c7021-fec1-4d5e-8aba-5a9f5f2cac2e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8967641dd624dacb5ff12c8e22599bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:54:01.915762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c7021-fec1-4d5e-8aba-5a9f5f2cac2e service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:01.916042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c7021-fec1-4d5e-8aba-5a9f5f2cac2e service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:01.916314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c7021-fec1-4d5e-8aba-5a9f5f2cac2e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:01.917941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ba3c7021-fec1-4d5e-8aba-5a9f5f2cac2e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:01.970970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:02.025034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c7021-fec1-4d5e-8aba-5a9f5f2cac2e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:54:02.026455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c7021-fec1-4d5e-8aba-5a9f5f2cac2e service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:54:02.045885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 730/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:54:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:54:04.128801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e204cf-954c-4fa5-a296-7e4437fa7e66 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:04.129276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e204cf-954c-4fa5-a296-7e4437fa7e66 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:04.129401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e204cf-954c-4fa5-a296-7e4437fa7e66 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:04.130792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 746/1476] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:54:04.136000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce13c6-7e9f-4999-8199-80c614edd199 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:04.136434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce13c6-7e9f-4999-8199-80c614edd199 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:04.136530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce13c6-7e9f-4999-8199-80c614edd199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:04.201101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d3ce13c6-7e9f-4999-8199-80c614edd199 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:54:04.202295 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d3ce13c6-7e9f-4999-8199-80c614edd199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:04.296012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 731/1477] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:54:04.319014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e204cf-954c-4fa5-a296-7e4437fa7e66 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:54:04.320550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c9e204cf-954c-4fa5-a296-7e4437fa7e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:04.447153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e204cf-954c-4fa5-a296-7e4437fa7e66 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:54:04.495328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:04.540066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2197e3-375c-4635-8365-85ee8e4868c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:54:04.540679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2197e3-375c-4635-8365-85ee8e4868c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:04.540954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2197e3-375c-4635-8365-85ee8e4868c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:04.541245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2197e3-375c-4635-8365-85ee8e4868c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:04.541935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2197e3-375c-4635-8365-85ee8e4868c1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:54:04.543672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-df2197e3-375c-4635-8365-85ee8e4868c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:54:04.543672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 28 03:54:04.543672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: Jul 28 03:54:04.544824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2197e3-375c-4635-8365-85ee8e4868c1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:54:04.563686 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 747/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jul 28 03:54:04 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:54:06.634575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec88c3-cfc3-4b12-b167-d1dd98de96d5 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:06.635174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec88c3-cfc3-4b12-b167-d1dd98de96d5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:06.635174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec88c3-cfc3-4b12-b167-d1dd98de96d5 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:06.636542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 732/1479] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:54:06.641539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bfd92-9054-4991-b789-1fe44fcad0f2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:06.641942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bfd92-9054-4991-b789-1fe44fcad0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:06.642086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bfd92-9054-4991-b789-1fe44fcad0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:06.708428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b36bfd92-9054-4991-b789-1fe44fcad0f2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:54:06.711161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b36bfd92-9054-4991-b789-1fe44fcad0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:06.806693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 748/1480] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:54:06.850184 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec88c3-cfc3-4b12-b167-d1dd98de96d5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:54:06.851950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bbec88c3-cfc3-4b12-b167-d1dd98de96d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:06.861487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:06.994802 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec88c3-cfc3-4b12-b167-d1dd98de96d5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:54:07.073445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:07.111935 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8e814-53d1-4912-a633-afd0de668bb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:54:07.112546 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8e814-53d1-4912-a633-afd0de668bb7 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:07.112823 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8e814-53d1-4912-a633-afd0de668bb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:07.113093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8e814-53d1-4912-a633-afd0de668bb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:07.114708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bec8e814-53d1-4912-a633-afd0de668bb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:07.205423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8e814-53d1-4912-a633-afd0de668bb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:54:07.207050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8e814-53d1-4912-a633-afd0de668bb7 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:54:07.225186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 733/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:54:09.395217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de6cf2-2557-4817-8074-0c85dfcf883a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:09.396017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de6cf2-2557-4817-8074-0c85dfcf883a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:09.396926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de6cf2-2557-4817-8074-0c85dfcf883a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:09.398511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 749/1482] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:54:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:54:09.403766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb83d4b-1322-4e14-8179-0041647736bc None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:09.404174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb83d4b-1322-4e14-8179-0041647736bc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:09.404329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb83d4b-1322-4e14-8179-0041647736bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:09.470924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6fb83d4b-1322-4e14-8179-0041647736bc None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:54:09.472124 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6fb83d4b-1322-4e14-8179-0041647736bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:09.517662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:09.569925 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 734/1483] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:54:09.606708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de6cf2-2557-4817-8074-0c85dfcf883a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:54:09.608260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-98de6cf2-2557-4817-8074-0c85dfcf883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:09.632378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:09.713108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de6cf2-2557-4817-8074-0c85dfcf883a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:54:09.785188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18809188-6818-471b-b73b-cabd150c1a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:54:09.785811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18809188-6818-471b-b73b-cabd150c1a67 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:09.786094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18809188-6818-471b-b73b-cabd150c1a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:09.786372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18809188-6818-471b-b73b-cabd150c1a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:09.788422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-18809188-6818-471b-b73b-cabd150c1a67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:09.861558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:09.896859 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18809188-6818-471b-b73b-cabd150c1a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:54:09.898294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18809188-6818-471b-b73b-cabd150c1a67 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:54:09.916407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 750/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:54:11.970877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d81526-b9a9-4616-ad39-086337d6c8dd None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:11.971418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d81526-b9a9-4616-ad39-086337d6c8dd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:11.971418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d81526-b9a9-4616-ad39-086337d6c8dd None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:11.972781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 735/1485] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:54:11.978716 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a7db7-ff0b-43c8-942b-eef5b55077fd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:11.979183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a7db7-ff0b-43c8-942b-eef5b55077fd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:11.979288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a7db7-ff0b-43c8-942b-eef5b55077fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:12.045519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8b9a7db7-ff0b-43c8-942b-eef5b55077fd None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:54:12.046719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8b9a7db7-ff0b-43c8-942b-eef5b55077fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:12.144669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 751/1486] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:54:12.183056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d81526-b9a9-4616-ad39-086337d6c8dd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:54:12.184587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-64d81526-b9a9-4616-ad39-086337d6c8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:12.288514 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d81526-b9a9-4616-ad39-086337d6c8dd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:54:12.362205 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:12.364133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3490b-bd4d-4436-b317-2b8a0d3f1260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:54:12.365503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3490b-bd4d-4436-b317-2b8a0d3f1260 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:12.366003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3490b-bd4d-4436-b317-2b8a0d3f1260 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:12.366454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3490b-bd4d-4436-b317-2b8a0d3f1260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:12.368202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7ad3490b-bd4d-4436-b317-2b8a0d3f1260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:12.476113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3490b-bd4d-4436-b317-2b8a0d3f1260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:54:12.477545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3490b-bd4d-4436-b317-2b8a0d3f1260 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:54:12.495477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 736/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:54:14.600379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6e0cc-f1cb-4900-ba3c-9df8ce03cd40 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:14.600898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6e0cc-f1cb-4900-ba3c-9df8ce03cd40 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:14.600898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6e0cc-f1cb-4900-ba3c-9df8ce03cd40 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:14.602258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 752/1488] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:54:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:54:14.607621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c154267-0ce2-42d1-8438-4c2fe1a52664 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:14.608049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c154267-0ce2-42d1-8438-4c2fe1a52664 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:14.608183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c154267-0ce2-42d1-8438-4c2fe1a52664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:14.673140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3c154267-0ce2-42d1-8438-4c2fe1a52664 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:54:14.674332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3c154267-0ce2-42d1-8438-4c2fe1a52664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:14.726402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:14.769228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 737/1489] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:54:14.809272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6e0cc-f1cb-4900-ba3c-9df8ce03cd40 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:54:14.810799 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-aae6e0cc-f1cb-4900-ba3c-9df8ce03cd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:14.846622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:14.915298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6e0cc-f1cb-4900-ba3c-9df8ce03cd40 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:54:14.991043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81717533-af8b-4720-93a8-a9f875bf0648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:54:14.991929 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81717533-af8b-4720-93a8-a9f875bf0648 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:14.992365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81717533-af8b-4720-93a8-a9f875bf0648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:14.992638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81717533-af8b-4720-93a8-a9f875bf0648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:14.994267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-81717533-af8b-4720-93a8-a9f875bf0648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:15.078487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:15.107367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81717533-af8b-4720-93a8-a9f875bf0648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:54:15.108802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81717533-af8b-4720-93a8-a9f875bf0648 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:54:15.127497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 753/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:54:17.507153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda662f-68e2-46bc-8a29-2cdc249d5f33 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:17.507725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda662f-68e2-46bc-8a29-2cdc249d5f33 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:17.507725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda662f-68e2-46bc-8a29-2cdc249d5f33 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:17.509035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 738/1491] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jul 28 03:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:54:17.514243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b91450-9ce3-4d2f-8cd3-b53021059cfb None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:17.514625 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b91450-9ce3-4d2f-8cd3-b53021059cfb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:17.514796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b91450-9ce3-4d2f-8cd3-b53021059cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:17.580644 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b0b91450-9ce3-4d2f-8cd3-b53021059cfb None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:54:17.581834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b0b91450-9ce3-4d2f-8cd3-b53021059cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:17.678555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 754/1492] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jul 28 03:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:54:17.716278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda662f-68e2-46bc-8a29-2cdc249d5f33 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:54:17.717857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-deda662f-68e2-46bc-8a29-2cdc249d5f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:17.820027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda662f-68e2-46bc-8a29-2cdc249d5f33 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:54:17.892361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91616e2b-08f7-4738-8eb6-57912f010352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:54:17.892971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91616e2b-08f7-4738-8eb6-57912f010352 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:54:17.893255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91616e2b-08f7-4738-8eb6-57912f010352 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:54:17.893527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91616e2b-08f7-4738-8eb6-57912f010352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:54:17.895179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-91616e2b-08f7-4738-8eb6-57912f010352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:54:17.904351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:18.005396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91616e2b-08f7-4738-8eb6-57912f010352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:54:18.006872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91616e2b-08f7-4738-8eb6-57912f010352 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:54:18.025639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 739/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:54:27.676064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=82 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:54:28.023675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=50 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:45.247513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702798ca-aea0-43f6-a01d-fecc3213aa40 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:45.248143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702798ca-aea0-43f6-a01d-fecc3213aa40 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:45.248143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702798ca-aea0-43f6-a01d-fecc3213aa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:45.326209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-702798ca-aea0-43f6-a01d-fecc3213aa40 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:55:45.327572 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-702798ca-aea0-43f6-a01d-fecc3213aa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:45.423413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 755/1494] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:55:45.432087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70a689-1abd-4ab7-8b9b-62dc4cefef92 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:45.432625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70a689-1abd-4ab7-8b9b-62dc4cefef92 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:45.432625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70a689-1abd-4ab7-8b9b-62dc4cefef92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:45.507702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9b70a689-1abd-4ab7-8b9b-62dc4cefef92 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:55:45.509073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9b70a689-1abd-4ab7-8b9b-62dc4cefef92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:45.607204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 740/1495] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:55:45.611811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702798ca-aea0-43f6-a01d-fecc3213aa40 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:55:45.613835 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-702798ca-aea0-43f6-a01d-fecc3213aa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:45.625096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702798ca-aea0-43f6-a01d-fecc3213aa40 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:55:45.686492 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:45.723371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1232ab7-25af-41bf-91fd-436021777d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:55:45.723987 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1232ab7-25af-41bf-91fd-436021777d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:45.724272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1232ab7-25af-41bf-91fd-436021777d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:45.724546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1232ab7-25af-41bf-91fd-436021777d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:45.725241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1232ab7-25af-41bf-91fd-436021777d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:55:45.725543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1232ab7-25af-41bf-91fd-436021777d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:55:45.726567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-f1232ab7-25af-41bf-91fd-436021777d6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:55:45.726567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 28 03:55:45.726567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: Jul 28 03:55:45.727549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1232ab7-25af-41bf-91fd-436021777d6b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:55:45.738753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 756/1496] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:55:45.747028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70a689-1abd-4ab7-8b9b-62dc4cefef92 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:55:45.755972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70a689-1abd-4ab7-8b9b-62dc4cefef92 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:55:45.815919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:45.849370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f838a-b791-4283-813a-195d577c592f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:55:45.850083 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f838a-b791-4283-813a-195d577c592f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:45.850366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f838a-b791-4283-813a-195d577c592f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:45.850634 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f838a-b791-4283-813a-195d577c592f admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:45.851329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f838a-b791-4283-813a-195d577c592f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:55:45.853049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-664f838a-b791-4283-813a-195d577c592f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:55:45.853049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 28 03:55:45.853049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: Jul 28 03:55:45.854229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f838a-b791-4283-813a-195d577c592f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:55:45.911258 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 741/1497] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:55:45 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:55:45.915665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1232ab7-25af-41bf-91fd-436021777d6b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:55:45.917216 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f1232ab7-25af-41bf-91fd-436021777d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:45.951278 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1232ab7-25af-41bf-91fd-436021777d6b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:55:46.043294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfd67b-0525-4c29-a2b0-9d41ccf9aa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:55:46.043920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd67b-0525-4c29-a2b0-9d41ccf9aa6a admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:46.044206 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd67b-0525-4c29-a2b0-9d41ccf9aa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:46.044485 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd67b-0525-4c29-a2b0-9d41ccf9aa6a admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:46.045194 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfd67b-0525-4c29-a2b0-9d41ccf9aa6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:55:46.046241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-51cfd67b-0525-4c29-a2b0-9d41ccf9aa6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:55:46.046241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 28 03:55:46.046241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: Jul 28 03:55:46.047497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfd67b-0525-4c29-a2b0-9d41ccf9aa6a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:55:46.091469 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 757/1498] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:55:45 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:55:46.095565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f838a-b791-4283-813a-195d577c592f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:55:46.097632 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-664f838a-b791-4283-813a-195d577c592f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:46.160444 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f838a-b791-4283-813a-195d577c592f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:55:46.242528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:46.253570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8b080-35c3-4f47-a509-6d6f9164a833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:55:46.254186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8b080-35c3-4f47-a509-6d6f9164a833 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:46.254469 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8b080-35c3-4f47-a509-6d6f9164a833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:46.254743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8b080-35c3-4f47-a509-6d6f9164a833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:46.255477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8b080-35c3-4f47-a509-6d6f9164a833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:55:46.256492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-30a8b080-35c3-4f47-a509-6d6f9164a833 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:55:46.256492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 28 03:55:46.256492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: Jul 28 03:55:46.257389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8b080-35c3-4f47-a509-6d6f9164a833 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:55:46.269546 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 742/1499] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:55:46.274489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfd67b-0525-4c29-a2b0-9d41ccf9aa6a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:55:46.276059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-51cfd67b-0525-4c29-a2b0-9d41ccf9aa6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:46.288313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfd67b-0525-4c29-a2b0-9d41ccf9aa6a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:55:46.313627 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:46.384007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa7e3b-4a30-43f6-9b34-6b647031ef2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:55:46.384551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa7e3b-4a30-43f6-9b34-6b647031ef2a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:46.384835 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa7e3b-4a30-43f6-9b34-6b647031ef2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:46.385106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa7e3b-4a30-43f6-9b34-6b647031ef2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:46.385805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa7e3b-4a30-43f6-9b34-6b647031ef2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:55:46.386846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-e2aa7e3b-4a30-43f6-9b34-6b647031ef2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:55:46.386846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 28 03:55:46.386846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: Jul 28 03:55:46.387740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa7e3b-4a30-43f6-9b34-6b647031ef2a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:55:46.400539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 758/1500] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:55:46.404740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8b080-35c3-4f47-a509-6d6f9164a833 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:55:46.406281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-30a8b080-35c3-4f47-a509-6d6f9164a833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:46.441056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8b080-35c3-4f47-a509-6d6f9164a833 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:55:46.535655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116eeef8-78a8-40b9-9326-9480d06e183d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:55:46.536329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116eeef8-78a8-40b9-9326-9480d06e183d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:46.536612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116eeef8-78a8-40b9-9326-9480d06e183d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:46.536891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116eeef8-78a8-40b9-9326-9480d06e183d admin admin] PATH_INFO: `/v3/projects/16609c89e8fb41459b78c45da341667d/users/ef6329fac09e460ea146c3fc478e27c2/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:46.560060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116eeef8-78a8-40b9-9326-9480d06e183d admin admin] RBAC: Authorizing `identity:create_grant(project_id=16609c89e8fb41459b78c45da341667d, user_id=ef6329fac09e460ea146c3fc478e27c2, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:55:46.561104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-116eeef8-78a8-40b9-9326-9480d06e183d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:55:46.561104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 28 03:55:46.561104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: Jul 28 03:55:46.563083 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116eeef8-78a8-40b9-9326-9480d06e183d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:55:46.590356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 743/1501] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:55:46 2022] PUT /identity/v3/projects/16609c89e8fb41459b78c45da341667d/users/ef6329fac09e460ea146c3fc478e27c2/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:55:46.602682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ffa03-3351-471d-8c75-6628a4b11e76 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:46.603003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ffa03-3351-471d-8c75-6628a4b11e76 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:46.603284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ffa03-3351-471d-8c75-6628a4b11e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:46.659478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-477ffa03-3351-471d-8c75-6628a4b11e76 None None] MFA Rules not processed for user `ef6329fac09e460ea146c3fc478e27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:55:46.660608 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-477ffa03-3351-471d-8c75-6628a4b11e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:46.737880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 759/1502] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:55:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:55:46.751938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca40de7-25d4-48c1-b8a6-f0212cdd5eec None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:46.752338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca40de7-25d4-48c1-b8a6-f0212cdd5eec None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:46.753191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca40de7-25d4-48c1-b8a6-f0212cdd5eec None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:46.754717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 744/1503] 10.222.0.46 () {54 vars in 964 bytes} [Thu Jul 28 03:55:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:55:46.759567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358e00-202a-473f-974a-ac4289817056 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:46.759889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358e00-202a-473f-974a-ac4289817056 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:46.760172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358e00-202a-473f-974a-ac4289817056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:46.792283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:46.837095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e2358e00-202a-473f-974a-ac4289817056 None None] MFA Rules not processed for user `e2cf2558517445c79e47c3b67501ca09`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:55:46.838289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e2358e00-202a-473f-974a-ac4289817056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:46.913884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 760/1504] 10.222.0.46 () {58 vars in 1061 bytes} [Thu Jul 28 03:55:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:55:46.921519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca40de7-25d4-48c1-b8a6-f0212cdd5eec None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:55:46.923078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-eca40de7-25d4-48c1-b8a6-f0212cdd5eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:46.975064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:47.029607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca40de7-25d4-48c1-b8a6-f0212cdd5eec None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:55:47.102658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8beca-3d0d-4390-b17e-88bd08440654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:55:47.103303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8beca-3d0d-4390-b17e-88bd08440654 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:47.103593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8beca-3d0d-4390-b17e-88bd08440654 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:47.103870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8beca-3d0d-4390-b17e-88bd08440654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:47.105504 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-33f8beca-3d0d-4390-b17e-88bd08440654 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:47.202844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8beca-3d0d-4390-b17e-88bd08440654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:55:47.204212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8beca-3d0d-4390-b17e-88bd08440654 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:55:47.208239 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:47.223750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 745/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:55:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:55:47.267946 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee4a2c-fc7f-478e-9652-85feb4da1d90 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:47.268776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee4a2c-fc7f-478e-9652-85feb4da1d90 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:47.269046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee4a2c-fc7f-478e-9652-85feb4da1d90 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:47.270333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 761/1506] 10.222.0.46 () {52 vars in 830 bytes} [Thu Jul 28 03:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:55:47.274014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8beca-3d0d-4390-b17e-88bd08440654 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:55:47.275275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-33f8beca-3d0d-4390-b17e-88bd08440654 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:47.373914 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8beca-3d0d-4390-b17e-88bd08440654 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:55:47.465696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:55:47.466354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:47.466517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:47.466807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] PATH_INFO: `/v3/users/ef6329fac09e460ea146c3fc478e27c2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:47.479320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6329fac09e460ea146c3fc478e27c2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:55:47.480339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:55:47.480339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 28 03:55:47.480339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: Jul 28 03:55:47.481536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:55:47.490995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:47.550545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6329fac09e460ea146c3fc478e27c2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:55:47.551113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6329fac09e460ea146c3fc478e27c2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:55:47.561828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6329fac09e460ea146c3fc478e27c2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:55:47.579728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6329fac09e460ea146c3fc478e27c2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:55:47.592325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6329fac09e460ea146c3fc478e27c2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:55:47.597802 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 746/1507] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 03:55:47 2022] DELETE /identity/v3/users/ef6329fac09e460ea146c3fc478e27c2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:55:47.602247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d47ab-571e-40f3-a01e-73b8e0dd547a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:47.602720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d47ab-571e-40f3-a01e-73b8e0dd547a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:47.602868 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d47ab-571e-40f3-a01e-73b8e0dd547a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:47.658513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9e6d47ab-571e-40f3-a01e-73b8e0dd547a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:55:47.659861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9e6d47ab-571e-40f3-a01e-73b8e0dd547a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:47.734986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:47.748921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 762/1508] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:55:47.759563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:55:47.760821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:47.860640 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656e083-7877-4778-baf7-f27f8a712b09 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:55:47.915398 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:47.936537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0aae9d-4f3a-48f9-b2f2-da64687d78e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:55:47.937168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aae9d-4f3a-48f9-b2f2-da64687d78e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:47.937459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aae9d-4f3a-48f9-b2f2-da64687d78e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:47.937739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aae9d-4f3a-48f9-b2f2-da64687d78e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:47.939415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4d0aae9d-4f3a-48f9-b2f2-da64687d78e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:48.048456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0aae9d-4f3a-48f9-b2f2-da64687d78e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:55:48.049902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0aae9d-4f3a-48f9-b2f2-da64687d78e7 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:55:48.068053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 747/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:55:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:55:48.412616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d47ab-571e-40f3-a01e-73b8e0dd547a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:55:48.414180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9e6d47ab-571e-40f3-a01e-73b8e0dd547a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:55:48.425854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d47ab-571e-40f3-a01e-73b8e0dd547a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:55:48.524196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:55:48.524864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:55:48.525148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:55:48.525427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] PATH_INFO: `/v3/projects/16609c89e8fb41459b78c45da341667d` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:55:48.536136 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] RBAC: Authorizing `identity:delete_project(project_id=16609c89e8fb41459b78c45da341667d)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:55:48.537179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:55:48.537179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 28 03:55:48.537179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: Jul 28 03:55:48.538643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:55:48.595548 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16609c89e8fb41459b78c45da341667d', 'request_id': 'req-600f1bf5-1d42-426b-9480-333bd7dc475c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:55:48.602044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16609c89e8fb41459b78c45da341667d', 'request_id': 'req-600f1bf5-1d42-426b-9480-333bd7dc475c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:55:48.616675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16609c89e8fb41459b78c45da341667d', 'request_id': 'req-600f1bf5-1d42-426b-9480-333bd7dc475c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:55:48.643590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] The token cache is being invalidate because project 16609c89e8fb41459b78c45da341667d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 03:55:48.643881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:55:48.644729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-600f1bf5-1d42-426b-9480-333bd7dc475c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:55:48.648195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 763/1510] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 03:55:48 2022] DELETE /identity/v3/projects/16609c89e8fb41459b78c45da341667d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:55:58.065605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=62 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:55:58.641029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:57:27.635384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a5c5a-ad90-4c74-a60d-87d22663d75d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:27.635971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a5c5a-ad90-4c74-a60d-87d22663d75d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:27.635971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a5c5a-ad90-4c74-a60d-87d22663d75d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:27.637296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:57:27.642184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34fbc0-3588-4e26-b55a-26ac3b65de49 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:27.642767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34fbc0-3588-4e26-b55a-26ac3b65de49 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:27.642945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34fbc0-3588-4e26-b55a-26ac3b65de49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:27.725198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5e34fbc0-3588-4e26-b55a-26ac3b65de49 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:57:27.726401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5e34fbc0-3588-4e26-b55a-26ac3b65de49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:27.828982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:57:27.987236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a5c5a-ad90-4c74-a60d-87d22663d75d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:57:27.989345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-603a5c5a-ad90-4c74-a60d-87d22663d75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:28.108301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a5c5a-ad90-4c74-a60d-87d22663d75d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:57:28.114294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14fc97-7e46-485a-b647-f6a335e7df47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:57:28.114944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14fc97-7e46-485a-b647-f6a335e7df47 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:28.115235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14fc97-7e46-485a-b647-f6a335e7df47 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:28.115515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14fc97-7e46-485a-b647-f6a335e7df47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:28.123499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14fc97-7e46-485a-b647-f6a335e7df47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:57:28.125086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14fc97-7e46-485a-b647-f6a335e7df47 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:57:28.134580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 749/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:57:28.983826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb2853-62a8-4972-a2e4-8e058c17880a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:28.984795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb2853-62a8-4972-a2e4-8e058c17880a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:28.984795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb2853-62a8-4972-a2e4-8e058c17880a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:28.985780 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:57:28.991263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973c151-6853-42b8-82f3-51648e720fc1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:28.991700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973c151-6853-42b8-82f3-51648e720fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:28.991818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973c151-6853-42b8-82f3-51648e720fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:29.012210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0973c151-6853-42b8-82f3-51648e720fc1 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:57:29.013400 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0973c151-6853-42b8-82f3-51648e720fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:29.021319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:57:29.173697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb2853-62a8-4972-a2e4-8e058c17880a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:57:29.188278 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb2853-62a8-4972-a2e4-8e058c17880a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:57:29.193850 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f29d8-9e82-48b1-822d-7a8e8910bf02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:57:29.194663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f29d8-9e82-48b1-822d-7a8e8910bf02 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:29.194977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f29d8-9e82-48b1-822d-7a8e8910bf02 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:29.195260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f29d8-9e82-48b1-822d-7a8e8910bf02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:29.205042 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f29d8-9e82-48b1-822d-7a8e8910bf02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:57:29.208306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f29d8-9e82-48b1-822d-7a8e8910bf02 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:57:29.218913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 766/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:57:29.743980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38577af1-d37d-483c-a8fd-df8cff444145 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:29.744261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38577af1-d37d-483c-a8fd-df8cff444145 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:29.744537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38577af1-d37d-483c-a8fd-df8cff444145 None None] PATH_INFO: `/v3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:29.745818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 751/1517] 10.222.0.39 () {54 vars in 925 bytes} [Thu Jul 28 03:57:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:57:29.752594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b32bcf-f04b-4c60-be5d-895eadefe295 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:29.752846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b32bcf-f04b-4c60-be5d-895eadefe295 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:29.753106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b32bcf-f04b-4c60-be5d-895eadefe295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:29.810732 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d4b32bcf-f04b-4c60-be5d-895eadefe295 None None] MFA Rules not processed for user `551281a31ce24e2f9b81b36f69419311`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:57:29.811989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d4b32bcf-f04b-4c60-be5d-895eadefe295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:29.828698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:57:29.884376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 767/1518] 10.222.0.39 () {58 vars in 1015 bytes} [Thu Jul 28 03:57:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:57:33.734109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a9994-7b72-447f-bfc5-ea4b9d3a20c5 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:33.734625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a9994-7b72-447f-bfc5-ea4b9d3a20c5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:33.734625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a9994-7b72-447f-bfc5-ea4b9d3a20c5 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:33.736625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 752/1519] 10.222.0.46 () {54 vars in 961 bytes} [Thu Jul 28 03:57:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:57:33.741478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13b51-ed4e-4f39-86d7-e38e1ab526c1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:33.741929 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13b51-ed4e-4f39-86d7-e38e1ab526c1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:33.742038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13b51-ed4e-4f39-86d7-e38e1ab526c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:33.761740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fcf13b51-ed4e-4f39-86d7-e38e1ab526c1 None None] MFA Rules not processed for user `551281a31ce24e2f9b81b36f69419311`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:57:33.762977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fcf13b51-ed4e-4f39-86d7-e38e1ab526c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:33.771619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 768/1520] 10.222.0.46 () {58 vars in 1058 bytes} [Thu Jul 28 03:57:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:57:33.779191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a9994-7b72-447f-bfc5-ea4b9d3a20c5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:57:33.792943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a9994-7b72-447f-bfc5-ea4b9d3a20c5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:57:33.797870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772e6b3-8c15-4511-bb77-0647efd1ca54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:57:33.799287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772e6b3-8c15-4511-bb77-0647efd1ca54 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:33.799578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772e6b3-8c15-4511-bb77-0647efd1ca54 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:33.799858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772e6b3-8c15-4511-bb77-0647efd1ca54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:33.807983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772e6b3-8c15-4511-bb77-0647efd1ca54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:57:33.809547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772e6b3-8c15-4511-bb77-0647efd1ca54 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:57:33.819709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 753/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:57:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:57:40.458334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bc135-997d-45c6-94c0-3d413e9012be None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:40.458927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bc135-997d-45c6-94c0-3d413e9012be None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:40.459028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bc135-997d-45c6-94c0-3d413e9012be None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:40.460341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:57:40.465080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460b23-a074-4728-96e3-81ab0e0f5a7f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:40.465505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460b23-a074-4728-96e3-81ab0e0f5a7f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:40.465620 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460b23-a074-4728-96e3-81ab0e0f5a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:40.485176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d7460b23-a074-4728-96e3-81ab0e0f5a7f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:57:40.487194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d7460b23-a074-4728-96e3-81ab0e0f5a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:40.494946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:57:40.656473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bc135-997d-45c6-94c0-3d413e9012be None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:57:40.671097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bc135-997d-45c6-94c0-3d413e9012be None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:57:40.677239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd565c5-1d75-4ca8-a5bb-0e846aec0f65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:57:40.677843 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd565c5-1d75-4ca8-a5bb-0e846aec0f65 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:40.678128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd565c5-1d75-4ca8-a5bb-0e846aec0f65 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:40.678407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd565c5-1d75-4ca8-a5bb-0e846aec0f65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:40.686415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd565c5-1d75-4ca8-a5bb-0e846aec0f65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:57:40.688036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd565c5-1d75-4ca8-a5bb-0e846aec0f65 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:57:40.697901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 770/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:57:41.513644 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615844d-209c-49af-a2e0-b8850572ae86 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:41.514187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615844d-209c-49af-a2e0-b8850572ae86 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:41.514187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615844d-209c-49af-a2e0-b8850572ae86 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:41.515581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 03:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:57:41.520818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab7d08-8900-4718-bb7f-de27eb228d75 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:41.521273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab7d08-8900-4718-bb7f-de27eb228d75 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:41.521381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab7d08-8900-4718-bb7f-de27eb228d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:41.542854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-03ab7d08-8900-4718-bb7f-de27eb228d75 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:57:41.544096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-03ab7d08-8900-4718-bb7f-de27eb228d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:41.552645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 03:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:57:41.708631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615844d-209c-49af-a2e0-b8850572ae86 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:57:41.723307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615844d-209c-49af-a2e0-b8850572ae86 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:57:41.728639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9945181-73ee-4c4f-b809-ede35c01abce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:57:41.729238 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9945181-73ee-4c4f-b809-ede35c01abce service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:41.729520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9945181-73ee-4c4f-b809-ede35c01abce service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:41.729796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9945181-73ee-4c4f-b809-ede35c01abce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:41.740834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9945181-73ee-4c4f-b809-ede35c01abce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:57:41.742428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9945181-73ee-4c4f-b809-ede35c01abce service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:57:41.752660 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 756/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:57:45.678213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab84fcc-7c8f-4e8b-b148-5884b013fa03 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:45.678803 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab84fcc-7c8f-4e8b-b148-5884b013fa03 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:45.678803 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab84fcc-7c8f-4e8b-b148-5884b013fa03 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:45.680140 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:57:45.685287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5940517-c176-4f89-ad14-4e83ab8c0071 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:45.685717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5940517-c176-4f89-ad14-4e83ab8c0071 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:45.685826 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5940517-c176-4f89-ad14-4e83ab8c0071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:45.706646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f5940517-c176-4f89-ad14-4e83ab8c0071 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:57:45.707872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f5940517-c176-4f89-ad14-4e83ab8c0071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:45.715608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:57:45.789642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab84fcc-7c8f-4e8b-b148-5884b013fa03 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:57:45.791338 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6ab84fcc-7c8f-4e8b-b148-5884b013fa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:45.869532 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab84fcc-7c8f-4e8b-b148-5884b013fa03 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:57:45.873243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88bd4e-1290-418f-8180-be2596027481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:57:45.873851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88bd4e-1290-418f-8180-be2596027481 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:45.874135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88bd4e-1290-418f-8180-be2596027481 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:45.874412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88bd4e-1290-418f-8180-be2596027481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:45.883351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88bd4e-1290-418f-8180-be2596027481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:57:45.884769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88bd4e-1290-418f-8180-be2596027481 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:57:45.894641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 773/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:57:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:57:47.889859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428790c-1d99-4136-93e2-3e0fee94fb80 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:47.890399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428790c-1d99-4136-93e2-3e0fee94fb80 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:47.890399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428790c-1d99-4136-93e2-3e0fee94fb80 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:47.892573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:57:47.897749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238276b-9640-4cf3-8980-f3fa23671db4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:47.898198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238276b-9640-4cf3-8980-f3fa23671db4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:47.898327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238276b-9640-4cf3-8980-f3fa23671db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:47.918392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3238276b-9640-4cf3-8980-f3fa23671db4 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:57:47.919604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3238276b-9640-4cf3-8980-f3fa23671db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:47.927395 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:57:47.963009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428790c-1d99-4136-93e2-3e0fee94fb80 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:57:47.974163 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428790c-1d99-4136-93e2-3e0fee94fb80 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:57:47.977810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a460d8-c2df-47ce-95c5-47d08f520075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:57:47.978417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a460d8-c2df-47ce-95c5-47d08f520075 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:47.978698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a460d8-c2df-47ce-95c5-47d08f520075 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:47.978992 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a460d8-c2df-47ce-95c5-47d08f520075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:47.987818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a460d8-c2df-47ce-95c5-47d08f520075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:57:47.989221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a460d8-c2df-47ce-95c5-47d08f520075 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:57:47.998343 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 759/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:57:51.516079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9541565-9a99-4b19-b879-2fe01559e9a5 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:51.516658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9541565-9a99-4b19-b879-2fe01559e9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:51.516760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9541565-9a99-4b19-b879-2fe01559e9a5 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:51.518565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:57:51.523701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcb2b1-ccf1-46a8-8c9f-a45caf9d4c03 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:51.524112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcb2b1-ccf1-46a8-8c9f-a45caf9d4c03 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:51.524263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcb2b1-ccf1-46a8-8c9f-a45caf9d4c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:51.531796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=81 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:57:51.545387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-39fcb2b1-ccf1-46a8-8c9f-a45caf9d4c03 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:57:51.546656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-39fcb2b1-ccf1-46a8-8c9f-a45caf9d4c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:51.554735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:57:51.858437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9541565-9a99-4b19-b879-2fe01559e9a5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:57:51.860186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b9541565-9a99-4b19-b879-2fe01559e9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:51.925557 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:57:51.939779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9541565-9a99-4b19-b879-2fe01559e9a5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:57:51.943375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ac194-97f1-4af3-8a4b-43a14ef41cef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:57:51.943990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac194-97f1-4af3-8a4b-43a14ef41cef service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:51.944273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac194-97f1-4af3-8a4b-43a14ef41cef service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:51.944551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac194-97f1-4af3-8a4b-43a14ef41cef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:51.953420 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ac194-97f1-4af3-8a4b-43a14ef41cef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:57:51.956103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ac194-97f1-4af3-8a4b-43a14ef41cef service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:57:51.965336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 776/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:57:54.046583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e363bf-5e81-49c2-8aae-5072099420d2 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:54.047147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e363bf-5e81-49c2-8aae-5072099420d2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:54.047274 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e363bf-5e81-49c2-8aae-5072099420d2 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:54.049566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 03:57:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:57:54.055097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1eaa7f-51e5-4185-a0b1-acc3d2d619bd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:54.055534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1eaa7f-51e5-4185-a0b1-acc3d2d619bd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:54.055656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1eaa7f-51e5-4185-a0b1-acc3d2d619bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:54.075948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-df1eaa7f-51e5-4185-a0b1-acc3d2d619bd None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:57:54.077154 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-df1eaa7f-51e5-4185-a0b1-acc3d2d619bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:54.085139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 03:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:57:54.395276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e363bf-5e81-49c2-8aae-5072099420d2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:57:54.396961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-72e363bf-5e81-49c2-8aae-5072099420d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:57:54.410288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e363bf-5e81-49c2-8aae-5072099420d2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:57:54.414010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06577db6-f9dd-4994-95dd-cd2896a5cc67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:57:54.414612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06577db6-f9dd-4994-95dd-cd2896a5cc67 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:57:54.414934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06577db6-f9dd-4994-95dd-cd2896a5cc67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:57:54.415211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06577db6-f9dd-4994-95dd-cd2896a5cc67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:57:54.424420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06577db6-f9dd-4994-95dd-cd2896a5cc67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:57:54.425831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06577db6-f9dd-4994-95dd-cd2896a5cc67 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:57:54.435212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 762/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:57:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:58:04.071052 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=11 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:58:04.431138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=8 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:23.499419 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e277418-23b3-449f-bd47-57fa7b8296d9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.502648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e277418-23b3-449f-bd47-57fa7b8296d9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.502648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e277418-23b3-449f-bd47-57fa7b8296d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.528035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5e277418-23b3-449f-bd47-57fa7b8296d9 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:23.529268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5e277418-23b3-449f-bd47-57fa7b8296d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:23.543390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 778/1540] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:23.557765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1096a-08ee-472e-bad4-8ad64e363024 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.558851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1096a-08ee-472e-bad4-8ad64e363024 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.559394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1096a-08ee-472e-bad4-8ad64e363024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.581516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b9c1096a-08ee-472e-bad4-8ad64e363024 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:23.583233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b9c1096a-08ee-472e-bad4-8ad64e363024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:23.594912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 763/1541] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:23.600606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e277418-23b3-449f-bd47-57fa7b8296d9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:23.621884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e277418-23b3-449f-bd47-57fa7b8296d9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:23.635535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c11a6-ec8f-495e-96aa-f576483e901c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:23.636181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c11a6-ec8f-495e-96aa-f576483e901c admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.636464 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c11a6-ec8f-495e-96aa-f576483e901c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.636744 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c11a6-ec8f-495e-96aa-f576483e901c admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.637466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c11a6-ec8f-495e-96aa-f576483e901c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:23.637776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c11a6-ec8f-495e-96aa-f576483e901c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:23.639938 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c11a6-ec8f-495e-96aa-f576483e901c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:23.653396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 779/1542] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:23.662084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1096a-08ee-472e-bad4-8ad64e363024 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:23.674823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f86a3e-6005-497f-872f-26f946a969f1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.674823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f86a3e-6005-497f-872f-26f946a969f1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.674823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f86a3e-6005-497f-872f-26f946a969f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.678013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1096a-08ee-472e-bad4-8ad64e363024 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:23.683096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351d30c-733b-4022-b864-7394b070435c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:23.683999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351d30c-733b-4022-b864-7394b070435c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.684509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351d30c-733b-4022-b864-7394b070435c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.685007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351d30c-733b-4022-b864-7394b070435c admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.686054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351d30c-733b-4022-b864-7394b070435c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:23.692474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351d30c-733b-4022-b864-7394b070435c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:23.697830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-97f86a3e-6005-497f-872f-26f946a969f1 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:23.699092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-97f86a3e-6005-497f-872f-26f946a969f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:23.716121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 780/1543] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:23.726818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6104eba-5909-496b-a244-41c64cec718b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.726818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6104eba-5909-496b-a244-41c64cec718b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.726818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6104eba-5909-496b-a244-41c64cec718b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.751473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 764/1544] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:23.752069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c6104eba-5909-496b-a244-41c64cec718b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:23.753637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351d30c-733b-4022-b864-7394b070435c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:23.754311 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c6104eba-5909-496b-a244-41c64cec718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:23.823928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351d30c-733b-4022-b864-7394b070435c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:23.829310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e03086-64c9-4a69-b0d0-66f7c32bffa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:23.830234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03086-64c9-4a69-b0d0-66f7c32bffa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.831408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03086-64c9-4a69-b0d0-66f7c32bffa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.832023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03086-64c9-4a69-b0d0-66f7c32bffa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.832930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e03086-64c9-4a69-b0d0-66f7c32bffa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:23.834315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-b9e03086-64c9-4a69-b0d0-66f7c32bffa8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:59:23.834315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 28 03:59:23.834315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: Jul 28 03:59:23.835975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e03086-64c9-4a69-b0d0-66f7c32bffa8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:23.853719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 781/1545] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:23.853719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edddf86-d3c0-49e5-be73-3a2682e0efc0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.853719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edddf86-d3c0-49e5-be73-3a2682e0efc0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.853719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edddf86-d3c0-49e5-be73-3a2682e0efc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.873338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 765/1546] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:23.874544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e03086-64c9-4a69-b0d0-66f7c32bffa8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:23.884944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6edddf86-d3c0-49e5-be73-3a2682e0efc0 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:23.884944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6edddf86-d3c0-49e5-be73-3a2682e0efc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:23.889836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e03086-64c9-4a69-b0d0-66f7c32bffa8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:23.895212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 782/1547] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:23.895502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784c510-7d6e-4d56-83a8-985f6cd8cf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:23.896257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c510-7d6e-4d56-83a8-985f6cd8cf7c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.896545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c510-7d6e-4d56-83a8-985f6cd8cf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.896827 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c510-7d6e-4d56-83a8-985f6cd8cf7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.897532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784c510-7d6e-4d56-83a8-985f6cd8cf7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:23.897892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784c510-7d6e-4d56-83a8-985f6cd8cf7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:23.898326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22214526-431f-432f-812b-7108a35ff8d7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.898874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22214526-431f-432f-812b-7108a35ff8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.898986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-e784c510-7d6e-4d56-83a8-985f6cd8cf7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:59:23.898986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 28 03:59:23.898986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: Jul 28 03:59:23.899645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22214526-431f-432f-812b-7108a35ff8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.899822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784c510-7d6e-4d56-83a8-985f6cd8cf7c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:23.918822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 766/1548] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:23.918822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b76d3-7eff-497f-b654-e68030a84b96 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.918822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b76d3-7eff-497f-b654-e68030a84b96 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.918822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b76d3-7eff-497f-b654-e68030a84b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.924919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-22214526-431f-432f-812b-7108a35ff8d7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:23.926188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-22214526-431f-432f-812b-7108a35ff8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:23.942600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 783/1549] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:23.944060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4ce7e-66df-4d9d-86bf-ec9ac153280e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.944354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4ce7e-66df-4d9d-86bf-ec9ac153280e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.944630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4ce7e-66df-4d9d-86bf-ec9ac153280e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.947311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1a0b76d3-7eff-497f-b654-e68030a84b96 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:23.948541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1a0b76d3-7eff-497f-b654-e68030a84b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:23.957949 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 767/1550] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:23.958729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b76d3-7eff-497f-b654-e68030a84b96 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:23.966346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-29b4ce7e-66df-4d9d-86bf-ec9ac153280e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:23.967595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-29b4ce7e-66df-4d9d-86bf-ec9ac153280e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:23.968654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b76d3-7eff-497f-b654-e68030a84b96 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:23.974355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0867dd-7384-47c3-949c-533061876f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:23.975004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0867dd-7384-47c3-949c-533061876f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.975313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0867dd-7384-47c3-949c-533061876f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.975587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0867dd-7384-47c3-949c-533061876f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.976287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0867dd-7384-47c3-949c-533061876f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:23.977396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 784/1551] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:23.978208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0867dd-7384-47c3-949c-533061876f73 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:23.978993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a7bb-03ba-4ef5-91f1-f6b91c1fe23e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:23.979553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a7bb-03ba-4ef5-91f1-f6b91c1fe23e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:23.979842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a7bb-03ba-4ef5-91f1-f6b91c1fe23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:23.990145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 768/1552] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:23.993996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0867dd-7384-47c3-949c-533061876f73 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.001456 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1a34a7bb-03ba-4ef5-91f1-f6b91c1fe23e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:24.002954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1a34a7bb-03ba-4ef5-91f1-f6b91c1fe23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:24.006539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0867dd-7384-47c3-949c-533061876f73 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.011079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 785/1553] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:24.011686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106dfdcd-e56d-4200-9600-d291a13c9844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.012365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebf340-e0ce-459c-b65d-913972927e80 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.012662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebf340-e0ce-459c-b65d-913972927e80 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.013075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebf340-e0ce-459c-b65d-913972927e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.013196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dfdcd-e56d-4200-9600-d291a13c9844 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.013358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dfdcd-e56d-4200-9600-d291a13c9844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.013637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dfdcd-e56d-4200-9600-d291a13c9844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.014301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106dfdcd-e56d-4200-9600-d291a13c9844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.016307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106dfdcd-e56d-4200-9600-d291a13c9844 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.032951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6eebf340-e0ce-459c-b65d-913972927e80 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:24.034701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6eebf340-e0ce-459c-b65d-913972927e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:24.043621 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 786/1554] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:24.044907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65beff2-5648-4403-a432-63188039b7b9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.045200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65beff2-5648-4403-a432-63188039b7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.045383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 769/1555] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:23 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:24.045563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65beff2-5648-4403-a432-63188039b7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.046646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814cdec-2c17-4628-89fe-0ae93f59d851 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.046971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814cdec-2c17-4628-89fe-0ae93f59d851 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.047255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814cdec-2c17-4628-89fe-0ae93f59d851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.066066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b65beff2-5648-4403-a432-63188039b7b9 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:24.067310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b65beff2-5648-4403-a432-63188039b7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:24.070390 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9814cdec-2c17-4628-89fe-0ae93f59d851 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:24.071581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9814cdec-2c17-4628-89fe-0ae93f59d851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:24.122049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:24.131161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 787/1556] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:24.131722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 770/1557] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:24.132430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e31303-5c0a-41d2-8ca1-2cbd4810b9fd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.132719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e31303-5c0a-41d2-8ca1-2cbd4810b9fd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.133000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e31303-5c0a-41d2-8ca1-2cbd4810b9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.133153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814cdec-2c17-4628-89fe-0ae93f59d851 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.145017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814cdec-2c17-4628-89fe-0ae93f59d851 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.150366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f38cf-561f-4c74-a7ef-24707db10a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.151010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f38cf-561f-4c74-a7ef-24707db10a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.151297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f38cf-561f-4c74-a7ef-24707db10a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.151573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f38cf-561f-4c74-a7ef-24707db10a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.152271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f38cf-561f-4c74-a7ef-24707db10a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.152837 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-51e31303-5c0a-41d2-8ca1-2cbd4810b9fd None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:24.153865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f38cf-561f-4c74-a7ef-24707db10a39 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.154033 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-51e31303-5c0a-41d2-8ca1-2cbd4810b9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:24.162048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 788/1558] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:24.163662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278d41a-4084-4619-ae72-9cc69d83b9c2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.163950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278d41a-4084-4619-ae72-9cc69d83b9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.164224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278d41a-4084-4619-ae72-9cc69d83b9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.167922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 771/1559] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:24.167922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f38cf-561f-4c74-a7ef-24707db10a39 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.176412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f38cf-561f-4c74-a7ef-24707db10a39 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.181373 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7fdf5-ff76-4a22-a43f-cad1fc541561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.181974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7fdf5-ff76-4a22-a43f-cad1fc541561 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.182258 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7fdf5-ff76-4a22-a43f-cad1fc541561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.182534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7fdf5-ff76-4a22-a43f-cad1fc541561 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.183257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7fdf5-ff76-4a22-a43f-cad1fc541561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:24.183378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6278d41a-4084-4619-ae72-9cc69d83b9c2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:24.183566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7fdf5-ff76-4a22-a43f-cad1fc541561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.184541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6278d41a-4084-4619-ae72-9cc69d83b9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:24.185145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7fdf5-ff76-4a22-a43f-cad1fc541561 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.194056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 789/1560] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:24.196194 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88051c4-e5ab-44e0-896b-bcebda194239 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.196673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88051c4-e5ab-44e0-896b-bcebda194239 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.197200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88051c4-e5ab-44e0-896b-bcebda194239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.200333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 772/1561] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:24.200333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7fdf5-ff76-4a22-a43f-cad1fc541561 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.216112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7fdf5-ff76-4a22-a43f-cad1fc541561 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.217176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e88051c4-e5ab-44e0-896b-bcebda194239 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:24.218337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e88051c4-e5ab-44e0-896b-bcebda194239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:24.221182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cd853-9f56-4c31-8b53-31407a88b67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.221912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cd853-9f56-4c31-8b53-31407a88b67f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.222247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cd853-9f56-4c31-8b53-31407a88b67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.222618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cd853-9f56-4c31-8b53-31407a88b67f admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.223365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cd853-9f56-4c31-8b53-31407a88b67f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.225261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cd853-9f56-4c31-8b53-31407a88b67f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.226174 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 790/1562] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:24.226973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88051c4-e5ab-44e0-896b-bcebda194239 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.236249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88051c4-e5ab-44e0-896b-bcebda194239 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.242395 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a21a16-8cf9-4c9e-af08-49e178bc6262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.243030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a21a16-8cf9-4c9e-af08-49e178bc6262 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.243314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a21a16-8cf9-4c9e-af08-49e178bc6262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.243590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a21a16-8cf9-4c9e-af08-49e178bc6262 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.244288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a21a16-8cf9-4c9e-af08-49e178bc6262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:24.244593 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a21a16-8cf9-4c9e-af08-49e178bc6262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.246187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a21a16-8cf9-4c9e-af08-49e178bc6262 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.246857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 773/1563] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/users => generated 292 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:24.248127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cd853-9f56-4c31-8b53-31407a88b67f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.262737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 791/1564] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:24.262737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a21a16-8cf9-4c9e-af08-49e178bc6262 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.271412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cd853-9f56-4c31-8b53-31407a88b67f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.272332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a21a16-8cf9-4c9e-af08-49e178bc6262 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.272427 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77188372-c526-475f-81f7-3544da5849c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.272427 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77188372-c526-475f-81f7-3544da5849c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.272427 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77188372-c526-475f-81f7-3544da5849c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.272427 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77188372-c526-475f-81f7-3544da5849c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.272768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77188372-c526-475f-81f7-3544da5849c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:24.272970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77188372-c526-475f-81f7-3544da5849c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.274566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77188372-c526-475f-81f7-3544da5849c9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.274983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad66a86-f127-4e1e-9686-d639f1c64917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.275585 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66a86-f127-4e1e-9686-d639f1c64917 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.275869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66a86-f127-4e1e-9686-d639f1c64917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.276150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66a86-f127-4e1e-9686-d639f1c64917 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.276841 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad66a86-f127-4e1e-9686-d639f1c64917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:24.277149 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad66a86-f127-4e1e-9686-d639f1c64917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.278723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad66a86-f127-4e1e-9686-d639f1c64917 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.281708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:24.287448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 774/1565] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:24.288512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77188372-c526-475f-81f7-3544da5849c9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.293677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 792/1566] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:24.295096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a88e-b12c-40b2-9816-5636391db302 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.295393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a88e-b12c-40b2-9816-5636391db302 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.295671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a88e-b12c-40b2-9816-5636391db302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.305040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77188372-c526-475f-81f7-3544da5849c9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.309729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282089b9-2b62-42b6-a7c8-a8acca5b7d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.310466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282089b9-2b62-42b6-a7c8-a8acca5b7d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.310754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282089b9-2b62-42b6-a7c8-a8acca5b7d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.311095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282089b9-2b62-42b6-a7c8-a8acca5b7d9a admin admin] PATH_INFO: `/v3/projects/18a927e9f5da42d7b34752c8b2db211a/users/c3308761e9d2495e95e127bbc7b9c5e2/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.316612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7b30a88e-b12c-40b2-9816-5636391db302 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:24.317830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7b30a88e-b12c-40b2-9816-5636391db302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:24.326104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 793/1567] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:24.326347 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282089b9-2b62-42b6-a7c8-a8acca5b7d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a927e9f5da42d7b34752c8b2db211a, user_id=c3308761e9d2495e95e127bbc7b9c5e2, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.326993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30a88e-b12c-40b2-9816-5636391db302 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.329068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282089b9-2b62-42b6-a7c8-a8acca5b7d9a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.336977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30a88e-b12c-40b2-9816-5636391db302 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.343116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 775/1568] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:24 2022] PUT /identity/v3/projects/18a927e9f5da42d7b34752c8b2db211a/users/c3308761e9d2495e95e127bbc7b9c5e2/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:24.343950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282089b9-2b62-42b6-a7c8-a8acca5b7d9a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.393950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297a306-6383-4635-a169-e3a23ab5a39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.394571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a306-6383-4635-a169-e3a23ab5a39a admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.394881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a306-6383-4635-a169-e3a23ab5a39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.395162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a306-6383-4635-a169-e3a23ab5a39a admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.395823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297a306-6383-4635-a169-e3a23ab5a39a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.396857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-9297a306-6383-4635-a169-e3a23ab5a39a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:59:24.396857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 28 03:59:24.396857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: Jul 28 03:59:24.398069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297a306-6383-4635-a169-e3a23ab5a39a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.404447 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282089b9-2b62-42b6-a7c8-a8acca5b7d9a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.409156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1af40-09e1-4b0e-942b-b6d0a6002c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.409759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1af40-09e1-4b0e-942b-b6d0a6002c48 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.410040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1af40-09e1-4b0e-942b-b6d0a6002c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.410310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1af40-09e1-4b0e-942b-b6d0a6002c48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.411035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1af40-09e1-4b0e-942b-b6d0a6002c48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:24.411451 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1af40-09e1-4b0e-942b-b6d0a6002c48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.413399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1af40-09e1-4b0e-942b-b6d0a6002c48 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.426180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 776/1569] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:24.426594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 794/1570] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:24.427036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1af40-09e1-4b0e-942b-b6d0a6002c48 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.427438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297a306-6383-4635-a169-e3a23ab5a39a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.436525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1af40-09e1-4b0e-942b-b6d0a6002c48 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.494362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665f15b-d07d-42fc-877c-cc3e43706ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.495192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665f15b-d07d-42fc-877c-cc3e43706ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.495586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665f15b-d07d-42fc-877c-cc3e43706ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.495966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665f15b-d07d-42fc-877c-cc3e43706ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.496128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297a306-6383-4635-a169-e3a23ab5a39a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.496745 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665f15b-d07d-42fc-877c-cc3e43706ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.498349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665f15b-d07d-42fc-877c-cc3e43706ef3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.500700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bd0f0-d7cc-4c17-8f4d-5d1d18c02220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.501306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bd0f0-d7cc-4c17-8f4d-5d1d18c02220 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.501587 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bd0f0-d7cc-4c17-8f4d-5d1d18c02220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.501861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bd0f0-d7cc-4c17-8f4d-5d1d18c02220 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.502754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bd0f0-d7cc-4c17-8f4d-5d1d18c02220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.504715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bd0f0-d7cc-4c17-8f4d-5d1d18c02220 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.508945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 777/1571] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:24.509985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665f15b-d07d-42fc-877c-cc3e43706ef3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.520874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665f15b-d07d-42fc-877c-cc3e43706ef3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.526081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8fcfa-2845-4eab-a951-fa84250e8542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.526869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8fcfa-2845-4eab-a951-fa84250e8542 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.527197 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8fcfa-2845-4eab-a951-fa84250e8542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.527565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8fcfa-2845-4eab-a951-fa84250e8542 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.528220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8fcfa-2845-4eab-a951-fa84250e8542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.530403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8fcfa-2845-4eab-a951-fa84250e8542 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.536414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 795/1572] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:24.536414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bd0f0-d7cc-4c17-8f4d-5d1d18c02220 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.545052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:24.567054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 778/1573] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:24.567887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8fcfa-2845-4eab-a951-fa84250e8542 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.605217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bd0f0-d7cc-4c17-8f4d-5d1d18c02220 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.608908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:24.629516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8fcfa-2845-4eab-a951-fa84250e8542 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.634035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bafc6b-949a-44cd-b817-47803671e241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.634644 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bafc6b-949a-44cd-b817-47803671e241 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.634956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bafc6b-949a-44cd-b817-47803671e241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.635228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bafc6b-949a-44cd-b817-47803671e241 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.636003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bafc6b-949a-44cd-b817-47803671e241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:24.636303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bafc6b-949a-44cd-b817-47803671e241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.637889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bafc6b-949a-44cd-b817-47803671e241 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.649144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 779/1574] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:24.650536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31883bf-2e6d-4a80-b9c3-9c3ea82c08f5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.650850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31883bf-2e6d-4a80-b9c3-9c3ea82c08f5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.651327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31883bf-2e6d-4a80-b9c3-9c3ea82c08f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.668085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20264c99-b683-4544-925b-1d7b39f564ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.668709 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20264c99-b683-4544-925b-1d7b39f564ed admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.668988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20264c99-b683-4544-925b-1d7b39f564ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.669267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20264c99-b683-4544-925b-1d7b39f564ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.669931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20264c99-b683-4544-925b-1d7b39f564ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.670494 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a31883bf-2e6d-4a80-b9c3-9c3ea82c08f5 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:24.671763 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a31883bf-2e6d-4a80-b9c3-9c3ea82c08f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:24.671871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20264c99-b683-4544-925b-1d7b39f564ed admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.679729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 780/1575] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:24.680485 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31883bf-2e6d-4a80-b9c3-9c3ea82c08f5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.690620 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31883bf-2e6d-4a80-b9c3-9c3ea82c08f5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.695513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced27de1-66a0-42c3-9c61-c4493e1a4db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.696221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced27de1-66a0-42c3-9c61-c4493e1a4db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.696520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced27de1-66a0-42c3-9c61-c4493e1a4db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.696837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced27de1-66a0-42c3-9c61-c4493e1a4db2 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.697525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced27de1-66a0-42c3-9c61-c4493e1a4db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.699654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced27de1-66a0-42c3-9c61-c4493e1a4db2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.709055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 796/1576] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/projects => generated 393 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:24.709055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20264c99-b683-4544-925b-1d7b39f564ed admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.726210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 781/1577] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/users => generated 285 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:24.726210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced27de1-66a0-42c3-9c61-c4493e1a4db2 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.733557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced27de1-66a0-42c3-9c61-c4493e1a4db2 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.772351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20264c99-b683-4544-925b-1d7b39f564ed admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.776942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c720a5c-a763-48ed-809f-7d39754e59e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.777557 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c720a5c-a763-48ed-809f-7d39754e59e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.777838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c720a5c-a763-48ed-809f-7d39754e59e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.778225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c720a5c-a763-48ed-809f-7d39754e59e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.778951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c720a5c-a763-48ed-809f-7d39754e59e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.780863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c720a5c-a763-48ed-809f-7d39754e59e5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.791331 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a28b71-412d-41ee-9401-8b094c01b7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.791956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a28b71-412d-41ee-9401-8b094c01b7fd admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.792244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a28b71-412d-41ee-9401-8b094c01b7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.792525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a28b71-412d-41ee-9401-8b094c01b7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.793231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a28b71-412d-41ee-9401-8b094c01b7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.794852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a28b71-412d-41ee-9401-8b094c01b7fd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.808198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 782/1578] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:24.808198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a28b71-412d-41ee-9401-8b094c01b7fd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.808492 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 797/1579] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:24.809278 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c720a5c-a763-48ed-809f-7d39754e59e5 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.820896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a28b71-412d-41ee-9401-8b094c01b7fd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.871636 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c720a5c-a763-48ed-809f-7d39754e59e5 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.871844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:24.876571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649ba79-ef38-415a-a26d-933aef582d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.877878 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649ba79-ef38-415a-a26d-933aef582d62 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.878481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649ba79-ef38-415a-a26d-933aef582d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.878582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649ba79-ef38-415a-a26d-933aef582d62 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.879289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649ba79-ef38-415a-a26d-933aef582d62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.880370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa379-ab44-4351-8f97-d81b54e035a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.880983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa379-ab44-4351-8f97-d81b54e035a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.881219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649ba79-ef38-415a-a26d-933aef582d62 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.881319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa379-ab44-4351-8f97-d81b54e035a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.881518 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa379-ab44-4351-8f97-d81b54e035a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.882175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaa379-ab44-4351-8f97-d81b54e035a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.884094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaa379-ab44-4351-8f97-d81b54e035a2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.903570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 798/1580] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:24.904517 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649ba79-ef38-415a-a26d-933aef582d62 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.909243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 783/1581] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:24.909243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa379-ab44-4351-8f97-d81b54e035a2 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.915623 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649ba79-ef38-415a-a26d-933aef582d62 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:24.920427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34a7f9-7491-46b5-92ef-c9912b7b0b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:24.921035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34a7f9-7491-46b5-92ef-c9912b7b0b6c admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:24.921322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34a7f9-7491-46b5-92ef-c9912b7b0b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:24.921599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34a7f9-7491-46b5-92ef-c9912b7b0b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:24.922257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34a7f9-7491-46b5-92ef-c9912b7b0b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:24.924319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34a7f9-7491-46b5-92ef-c9912b7b0b6c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:24.928799 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:24.956840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 799/1582] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:24.956840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34a7f9-7491-46b5-92ef-c9912b7b0b6c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:24.964486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa379-ab44-4351-8f97-d81b54e035a2 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.013372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f71bdb-f8c9-408b-9346-ec3fb913739f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.013994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f71bdb-f8c9-408b-9346-ec3fb913739f service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.014284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f71bdb-f8c9-408b-9346-ec3fb913739f service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.014559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f71bdb-f8c9-408b-9346-ec3fb913739f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.016272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34a7f9-7491-46b5-92ef-c9912b7b0b6c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.021645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b40a64-2228-4dbf-86fd-200482ecc232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.022252 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40a64-2228-4dbf-86fd-200482ecc232 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.022597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40a64-2228-4dbf-86fd-200482ecc232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.022825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40a64-2228-4dbf-86fd-200482ecc232 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.023484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b40a64-2228-4dbf-86fd-200482ecc232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.025051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f71bdb-f8c9-408b-9346-ec3fb913739f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.025383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b40a64-2228-4dbf-86fd-200482ecc232 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.026521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f71bdb-f8c9-408b-9346-ec3fb913739f service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.038002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 784/1583] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:25.039257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f71bdb-f8c9-408b-9346-ec3fb913739f service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.047708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 800/1584] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:24 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:25.048513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b40a64-2228-4dbf-86fd-200482ecc232 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.049121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f71bdb-f8c9-408b-9346-ec3fb913739f service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.053933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbecc106-47f7-418a-871e-fdddb0e0e131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.054544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecc106-47f7-418a-871e-fdddb0e0e131 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.054849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecc106-47f7-418a-871e-fdddb0e0e131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.055127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecc106-47f7-418a-871e-fdddb0e0e131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.055819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbecc106-47f7-418a-871e-fdddb0e0e131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.057412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbecc106-47f7-418a-871e-fdddb0e0e131 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.060059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b40a64-2228-4dbf-86fd-200482ecc232 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.066027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab79baf-2966-4ae8-a9c7-9187c2942318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.066700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab79baf-2966-4ae8-a9c7-9187c2942318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.067030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab79baf-2966-4ae8-a9c7-9187c2942318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.067308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab79baf-2966-4ae8-a9c7-9187c2942318 admin admin] PATH_INFO: `/v3/projects/726a1ad14d374061a5db26470f86fdf7/users/4dab551c284242df98e62efdc5de8de7/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.067847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 785/1585] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.068726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbecc106-47f7-418a-871e-fdddb0e0e131 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.081779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbecc106-47f7-418a-871e-fdddb0e0e131 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.086481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e7e4c-e735-4574-9db6-1c22be154a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.087127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e7e4c-e735-4574-9db6-1c22be154a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.087408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e7e4c-e735-4574-9db6-1c22be154a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.087677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e7e4c-e735-4574-9db6-1c22be154a05 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.088061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab79baf-2966-4ae8-a9c7-9187c2942318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=726a1ad14d374061a5db26470f86fdf7, user_id=4dab551c284242df98e62efdc5de8de7, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.088334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e7e4c-e735-4574-9db6-1c22be154a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.089300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-8ab79baf-2966-4ae8-a9c7-9187c2942318 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:59:25.089300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 28 03:59:25.089300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: Jul 28 03:59:25.090246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e7e4c-e735-4574-9db6-1c22be154a05 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.091282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab79baf-2966-4ae8-a9c7-9187c2942318 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.108597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 801/1586] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:25 2022] PUT /identity/v3/projects/726a1ad14d374061a5db26470f86fdf7/users/4dab551c284242df98e62efdc5de8de7/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:25.109816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab79baf-2966-4ae8-a9c7-9187c2942318 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.123166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 786/1587] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:25 2022] POST /identity/v3/users => generated 303 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:25.124218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e7e4c-e735-4574-9db6-1c22be154a05 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.134686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e7e4c-e735-4574-9db6-1c22be154a05 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.197310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab79baf-2966-4ae8-a9c7-9187c2942318 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.202445 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707e254-61e9-463a-a16a-4e95054addb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.203115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707e254-61e9-463a-a16a-4e95054addb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.203454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707e254-61e9-463a-a16a-4e95054addb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.203650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707e254-61e9-463a-a16a-4e95054addb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.204347 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707e254-61e9-463a-a16a-4e95054addb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.206333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707e254-61e9-463a-a16a-4e95054addb5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.206985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75458f7b-e2ab-476c-bd86-baf0dc24135a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.208340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458f7b-e2ab-476c-bd86-baf0dc24135a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.208752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458f7b-e2ab-476c-bd86-baf0dc24135a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.209037 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458f7b-e2ab-476c-bd86-baf0dc24135a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.209750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75458f7b-e2ab-476c-bd86-baf0dc24135a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.212039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75458f7b-e2ab-476c-bd86-baf0dc24135a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.216773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 802/1588] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.217588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707e254-61e9-463a-a16a-4e95054addb5 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.223940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 787/1589] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.225336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75458f7b-e2ab-476c-bd86-baf0dc24135a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.232317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707e254-61e9-463a-a16a-4e95054addb5 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.235937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75458f7b-e2ab-476c-bd86-baf0dc24135a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.241272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c8489-a2dd-4714-a0f2-a08e400ae624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.241434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c8489-a2dd-4714-a0f2-a08e400ae624 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.241717 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c8489-a2dd-4714-a0f2-a08e400ae624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.241991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c8489-a2dd-4714-a0f2-a08e400ae624 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.244865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c8489-a2dd-4714-a0f2-a08e400ae624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.245726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c8489-a2dd-4714-a0f2-a08e400ae624 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.247386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9dd0f3-dbc7-48c7-a93d-e40f388a2b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.248198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dd0f3-dbc7-48c7-a93d-e40f388a2b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.248314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dd0f3-dbc7-48c7-a93d-e40f388a2b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.248608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dd0f3-dbc7-48c7-a93d-e40f388a2b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.249434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9dd0f3-dbc7-48c7-a93d-e40f388a2b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.251021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9dd0f3-dbc7-48c7-a93d-e40f388a2b1a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.271218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:25.271379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 803/1590] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:25 2022] POST /identity/v3/users => generated 286 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:25.275460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c8489-a2dd-4714-a0f2-a08e400ae624 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.281004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 788/1591] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.281004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8490f7-3a9c-4b92-9434-7ea5944f65a4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.281711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8490f7-3a9c-4b92-9434-7ea5944f65a4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.282274 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8490f7-3a9c-4b92-9434-7ea5944f65a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.287633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c8489-a2dd-4714-a0f2-a08e400ae624 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.293190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffe977-c306-49ba-887d-61aad9cf55d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.293808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffe977-c306-49ba-887d-61aad9cf55d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.294090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffe977-c306-49ba-887d-61aad9cf55d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.294364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffe977-c306-49ba-887d-61aad9cf55d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.295238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dffe977-c306-49ba-887d-61aad9cf55d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.297055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dffe977-c306-49ba-887d-61aad9cf55d2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.310775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 804/1592] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.311870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffe977-c306-49ba-887d-61aad9cf55d2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.312965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ed8490f7-3a9c-4b92-9434-7ea5944f65a4 None None] MFA Rules not processed for user `4dab551c284242df98e62efdc5de8de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:25.315421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ed8490f7-3a9c-4b92-9434-7ea5944f65a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:25.324323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffe977-c306-49ba-887d-61aad9cf55d2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.331121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d691a3f-bffd-4290-9f6c-63c914fe2c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.331758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d691a3f-bffd-4290-9f6c-63c914fe2c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.332042 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d691a3f-bffd-4290-9f6c-63c914fe2c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.332318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d691a3f-bffd-4290-9f6c-63c914fe2c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.333016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d691a3f-bffd-4290-9f6c-63c914fe2c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.334615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d691a3f-bffd-4290-9f6c-63c914fe2c89 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.347336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 805/1593] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.348421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d691a3f-bffd-4290-9f6c-63c914fe2c89 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.360736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d691a3f-bffd-4290-9f6c-63c914fe2c89 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.366418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8114d1-8c12-417e-9620-65b787eec416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.367069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8114d1-8c12-417e-9620-65b787eec416 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.367356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8114d1-8c12-417e-9620-65b787eec416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.367632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8114d1-8c12-417e-9620-65b787eec416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.368660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8114d1-8c12-417e-9620-65b787eec416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.370421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8114d1-8c12-417e-9620-65b787eec416 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.371617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:25.387136 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 806/1594] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.388087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8114d1-8c12-417e-9620-65b787eec416 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.398519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 789/1595] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:25.399426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8114d1-8c12-417e-9620-65b787eec416 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.399841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8490f7-3a9c-4b92-9434-7ea5944f65a4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.407834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb48c62-0747-4783-812d-0bbdd9aae525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.409010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb48c62-0747-4783-812d-0bbdd9aae525 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.409295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb48c62-0747-4783-812d-0bbdd9aae525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.409571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb48c62-0747-4783-812d-0bbdd9aae525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.410266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb48c62-0747-4783-812d-0bbdd9aae525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.411949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb48c62-0747-4783-812d-0bbdd9aae525 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.414313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8490f7-3a9c-4b92-9434-7ea5944f65a4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.418927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b84c7-7e64-49b3-97ec-73b0a29de454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.419602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b84c7-7e64-49b3-97ec-73b0a29de454 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.419850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b84c7-7e64-49b3-97ec-73b0a29de454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.420094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b84c7-7e64-49b3-97ec-73b0a29de454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.420792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b84c7-7e64-49b3-97ec-73b0a29de454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.422370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b84c7-7e64-49b3-97ec-73b0a29de454 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.428996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 807/1596] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.430212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb48c62-0747-4783-812d-0bbdd9aae525 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.435430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 790/1597] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.436206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b84c7-7e64-49b3-97ec-73b0a29de454 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.447892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb48c62-0747-4783-812d-0bbdd9aae525 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.450480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b84c7-7e64-49b3-97ec-73b0a29de454 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.454308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97baa9-4942-4c04-92f8-1da973f7c1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.455626 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97baa9-4942-4c04-92f8-1da973f7c1da admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.455992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97baa9-4942-4c04-92f8-1da973f7c1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.456303 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97baa9-4942-4c04-92f8-1da973f7c1da admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.456888 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9acbd1-ade8-4b8b-9b65-dfe1b8ce0721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.457036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97baa9-4942-4c04-92f8-1da973f7c1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.457967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9acbd1-ade8-4b8b-9b65-dfe1b8ce0721 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.458255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9acbd1-ade8-4b8b-9b65-dfe1b8ce0721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.458580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9acbd1-ade8-4b8b-9b65-dfe1b8ce0721 admin admin] PATH_INFO: `/v3/projects/6cac357ba49f45b8b4305db5fe1b5aa9/users/46300b1878604dabbe36d06ede537258/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.458700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97baa9-4942-4c04-92f8-1da973f7c1da admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.473919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 808/1598] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.475059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97baa9-4942-4c04-92f8-1da973f7c1da admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.479217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9acbd1-ade8-4b8b-9b65-dfe1b8ce0721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cac357ba49f45b8b4305db5fe1b5aa9, user_id=46300b1878604dabbe36d06ede537258, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.482965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9acbd1-ade8-4b8b-9b65-dfe1b8ce0721 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.488192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97baa9-4942-4c04-92f8-1da973f7c1da admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.494486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e997c8c-dadb-4f9d-ba92-21897198496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.495392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e997c8c-dadb-4f9d-ba92-21897198496b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.495777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e997c8c-dadb-4f9d-ba92-21897198496b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.496063 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e997c8c-dadb-4f9d-ba92-21897198496b admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.496783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e997c8c-dadb-4f9d-ba92-21897198496b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.498563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e997c8c-dadb-4f9d-ba92-21897198496b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.503520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 791/1599] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:25 2022] PUT /identity/v3/projects/6cac357ba49f45b8b4305db5fe1b5aa9/users/46300b1878604dabbe36d06ede537258/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:25.508330 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9acbd1-ade8-4b8b-9b65-dfe1b8ce0721 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.521690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 809/1600] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.522539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e997c8c-dadb-4f9d-ba92-21897198496b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.541907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e997c8c-dadb-4f9d-ba92-21897198496b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.603863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8252274-d3d9-4036-a507-fa99c18735cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.604888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8252274-d3d9-4036-a507-fa99c18735cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.604888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8252274-d3d9-4036-a507-fa99c18735cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.605098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8252274-d3d9-4036-a507-fa99c18735cb admin admin] PATH_INFO: `/v3/projects/5db0f08a14ac472fb3edf2107c9ccfc5/users/7112379fb6794027a7c1519b2f77a5d4/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.613135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9acbd1-ade8-4b8b-9b65-dfe1b8ce0721 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.618387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411af02-828a-42ec-a2b0-43e4c133cf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.619119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411af02-828a-42ec-a2b0-43e4c133cf0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.619426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411af02-828a-42ec-a2b0-43e4c133cf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.619708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411af02-828a-42ec-a2b0-43e4c133cf0d admin admin] PATH_INFO: `/v3/projects/cb2697d81b164ddfbb33acd843f461f6/users/d09411d75019428a922e12e02c72f4af/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.621225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8252274-d3d9-4036-a507-fa99c18735cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db0f08a14ac472fb3edf2107c9ccfc5, user_id=7112379fb6794027a7c1519b2f77a5d4, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.624095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8252274-d3d9-4036-a507-fa99c18735cb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.638416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 810/1601] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:25 2022] PUT /identity/v3/projects/5db0f08a14ac472fb3edf2107c9ccfc5/users/7112379fb6794027a7c1519b2f77a5d4/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:25.639244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411af02-828a-42ec-a2b0-43e4c133cf0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2697d81b164ddfbb33acd843f461f6, user_id=d09411d75019428a922e12e02c72f4af, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.639869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7fc2c-b2fc-40d9-ba86-230fd2ef660e None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.640223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7fc2c-b2fc-40d9-ba86-230fd2ef660e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.640567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7fc2c-b2fc-40d9-ba86-230fd2ef660e None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.643437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 811/1602] 10.222.0.46 () {54 vars in 962 bytes} [Thu Jul 28 03:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:59:25.644296 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7fc2c-b2fc-40d9-ba86-230fd2ef660e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.645216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411af02-828a-42ec-a2b0-43e4c133cf0d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.661915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 792/1603] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:25 2022] PUT /identity/v3/projects/cb2697d81b164ddfbb33acd843f461f6/users/d09411d75019428a922e12e02c72f4af/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:25.663847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411af02-828a-42ec-a2b0-43e4c133cf0d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.713253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:25.723079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7fc2c-b2fc-40d9-ba86-230fd2ef660e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.737737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411af02-828a-42ec-a2b0-43e4c133cf0d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.744561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54738a7a-8b13-4294-9cc4-f8b2622b066d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.745246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738a7a-8b13-4294-9cc4-f8b2622b066d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.745624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738a7a-8b13-4294-9cc4-f8b2622b066d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.746006 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738a7a-8b13-4294-9cc4-f8b2622b066d admin admin] PATH_INFO: `/v3/projects/6ca5212b1a1743f2910407d7b8ba7aaf/users/72dbbd19e4454bee84fe5bc797497bd0/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.763871 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54738a7a-8b13-4294-9cc4-f8b2622b066d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca5212b1a1743f2910407d7b8ba7aaf, user_id=72dbbd19e4454bee84fe5bc797497bd0, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.766379 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54738a7a-8b13-4294-9cc4-f8b2622b066d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.775488 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:25.784927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 793/1604] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:25 2022] PUT /identity/v3/projects/6ca5212b1a1743f2910407d7b8ba7aaf/users/72dbbd19e4454bee84fe5bc797497bd0/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:25.786262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40371b0-8db6-45ad-b1db-5661ad4345f8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.786766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40371b0-8db6-45ad-b1db-5661ad4345f8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.787289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40371b0-8db6-45ad-b1db-5661ad4345f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.790999 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b574a8-1441-48e4-895d-41153c8d2558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.791628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b574a8-1441-48e4-895d-41153c8d2558 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.791905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b574a8-1441-48e4-895d-41153c8d2558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.792281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b574a8-1441-48e4-895d-41153c8d2558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.796499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b574a8-1441-48e4-895d-41153c8d2558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.796499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b574a8-1441-48e4-895d-41153c8d2558 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.816695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 812/1605] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:25.816695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b574a8-1441-48e4-895d-41153c8d2558 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:25.826207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b574a8-1441-48e4-895d-41153c8d2558 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:25.836550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e40371b0-8db6-45ad-b1db-5661ad4345f8 None None] MFA Rules not processed for user `46300b1878604dabbe36d06ede537258`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:25.838344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e40371b0-8db6-45ad-b1db-5661ad4345f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:25.907796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fc7f8-0f65-4de8-9f67-220658f580e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:25.909328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fc7f8-0f65-4de8-9f67-220658f580e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.909619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fc7f8-0f65-4de8-9f67-220658f580e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.909934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fc7f8-0f65-4de8-9f67-220658f580e4 admin admin] PATH_INFO: `/v3/projects/d04e2d7f19d6459899ccb47d0668b22f/users/da9842e3910641e7851c50ad541ea569/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.923082 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 794/1606] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:25.924900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15bab8-73b0-45d3-bd6a-ac8be5de4eb7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.925288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15bab8-73b0-45d3-bd6a-ac8be5de4eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.925565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15bab8-73b0-45d3-bd6a-ac8be5de4eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:25.929551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fc7f8-0f65-4de8-9f67-220658f580e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d04e2d7f19d6459899ccb47d0668b22f, user_id=da9842e3910641e7851c50ad541ea569, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:25.932579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fc7f8-0f65-4de8-9f67-220658f580e4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:25.949229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3d15bab8-73b0-45d3-bd6a-ac8be5de4eb7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:25.950336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3d15bab8-73b0-45d3-bd6a-ac8be5de4eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:25.952795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 813/1607] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:25 2022] PUT /identity/v3/projects/d04e2d7f19d6459899ccb47d0668b22f/users/da9842e3910641e7851c50ad541ea569/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:25.954284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705a6fc-76b4-43d2-8882-75b99518fb78 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:25.954574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705a6fc-76b4-43d2-8882-75b99518fb78 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:25.955200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705a6fc-76b4-43d2-8882-75b99518fb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.011366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c705a6fc-76b4-43d2-8882-75b99518fb78 None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:26.012725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c705a6fc-76b4-43d2-8882-75b99518fb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:26.040780 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 795/1608] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:26.042369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e7c3a-d545-43dc-bb84-719fb3a35132 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.042884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e7c3a-d545-43dc-bb84-719fb3a35132 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.051753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e7c3a-d545-43dc-bb84-719fb3a35132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.076109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8e3e7c3a-d545-43dc-bb84-719fb3a35132 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:26.078170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8e3e7c3a-d545-43dc-bb84-719fb3a35132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:26.088810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 796/1609] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:26.090832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bcc4c-d913-4d6d-8038-b9db8ed2f837 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.091175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bcc4c-d913-4d6d-8038-b9db8ed2f837 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.091446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bcc4c-d913-4d6d-8038-b9db8ed2f837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.116523 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6f0bcc4c-d913-4d6d-8038-b9db8ed2f837 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:26.117979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6f0bcc4c-d913-4d6d-8038-b9db8ed2f837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:26.126138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 797/1610] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:26.126397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 814/1611] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Jul 28 03:59:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:26.128087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705a6fc-76b4-43d2-8882-75b99518fb78 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:26.137573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bcc4c-d913-4d6d-8038-b9db8ed2f837 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:26.150095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bcc4c-d913-4d6d-8038-b9db8ed2f837 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:26.150489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705a6fc-76b4-43d2-8882-75b99518fb78 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:26.156443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bec40-0b20-4dbe-8a72-b358ccc95a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:26.157358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792221c-76eb-44e9-9db8-357e1a207417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:26.157692 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bec40-0b20-4dbe-8a72-b358ccc95a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.157971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bec40-0b20-4dbe-8a72-b358ccc95a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.158082 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792221c-76eb-44e9-9db8-357e1a207417 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.158240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bec40-0b20-4dbe-8a72-b358ccc95a1d admin admin] PATH_INFO: `/v3/projects/e26d0a638fa2418a834defc95b0f4c08/users/005ddb1d20df40d9897d4fa625fdb9f2/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.159033 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792221c-76eb-44e9-9db8-357e1a207417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.159365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792221c-76eb-44e9-9db8-357e1a207417 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.160280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792221c-76eb-44e9-9db8-357e1a207417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:26.162909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792221c-76eb-44e9-9db8-357e1a207417 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:26.177940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bec40-0b20-4dbe-8a72-b358ccc95a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26d0a638fa2418a834defc95b0f4c08, user_id=005ddb1d20df40d9897d4fa625fdb9f2, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:26.181739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bec40-0b20-4dbe-8a72-b358ccc95a1d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:26.195114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 815/1612] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:26 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:26.196597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279ea1-3bb9-4d33-8b50-ced3e484a6f8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.196941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279ea1-3bb9-4d33-8b50-ced3e484a6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.197280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279ea1-3bb9-4d33-8b50-ced3e484a6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.210980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 798/1613] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:26 2022] PUT /identity/v3/projects/e26d0a638fa2418a834defc95b0f4c08/users/005ddb1d20df40d9897d4fa625fdb9f2/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:26.210980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bec40-0b20-4dbe-8a72-b358ccc95a1d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:26.210980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-692bec40-0b20-4dbe-8a72-b358ccc95a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:26.231047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:26.242892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ed279ea1-3bb9-4d33-8b50-ced3e484a6f8 None None] MFA Rules not processed for user `da9842e3910641e7851c50ad541ea569`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:26.244549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ed279ea1-3bb9-4d33-8b50-ced3e484a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:26.265369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:26.289758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bec40-0b20-4dbe-8a72-b358ccc95a1d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:26.334670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 816/1614] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:26.335656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed279ea1-3bb9-4d33-8b50-ced3e484a6f8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:26.347412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd761d-8416-4234-83a9-a5127bc766d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:26.347917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd761d-8416-4234-83a9-a5127bc766d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.348201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd761d-8416-4234-83a9-a5127bc766d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.348533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd761d-8416-4234-83a9-a5127bc766d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.407431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed279ea1-3bb9-4d33-8b50-ced3e484a6f8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:26.412215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f561777-25ed-483d-bce7-95356ec33fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:26.412842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561777-25ed-483d-bce7-95356ec33fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.413384 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561777-25ed-483d-bce7-95356ec33fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.413661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561777-25ed-483d-bce7-95356ec33fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.418741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd761d-8416-4234-83a9-a5127bc766d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:26.421325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd761d-8416-4234-83a9-a5127bc766d6 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:26.423842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f561777-25ed-483d-bce7-95356ec33fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:26.426147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f561777-25ed-483d-bce7-95356ec33fe4 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:26.431993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 799/1615] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:26.432862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd761d-8416-4234-83a9-a5127bc766d6 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:26.438553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 817/1616] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:26.439558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f561777-25ed-483d-bce7-95356ec33fe4 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:26.460647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f561777-25ed-483d-bce7-95356ec33fe4 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:26.468930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ae60a-cb7b-4c45-95b6-d16cc550a509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:26.469449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ae60a-cb7b-4c45-95b6-d16cc550a509 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.469801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ae60a-cb7b-4c45-95b6-d16cc550a509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.470083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ae60a-cb7b-4c45-95b6-d16cc550a509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.493170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ae60a-cb7b-4c45-95b6-d16cc550a509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:26.494672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ae60a-cb7b-4c45-95b6-d16cc550a509 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:26.507470 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 818/1617] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:26.508402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ae60a-cb7b-4c45-95b6-d16cc550a509 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:26.525373 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ae60a-cb7b-4c45-95b6-d16cc550a509 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:26.536563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e281eb-0eff-467c-a0cc-12f02dfbbe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:26.537361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e281eb-0eff-467c-a0cc-12f02dfbbe02 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.538666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e281eb-0eff-467c-a0cc-12f02dfbbe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.538666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e281eb-0eff-467c-a0cc-12f02dfbbe02 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.539553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e281eb-0eff-467c-a0cc-12f02dfbbe02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:26.542036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e281eb-0eff-467c-a0cc-12f02dfbbe02 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:26.584805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd761d-8416-4234-83a9-a5127bc766d6 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:26.587782 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 819/1618] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:26 2022] POST /identity/v3/users => generated 290 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:26.589939 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee96b7d-2036-4103-9158-efc66cde3fe0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.590073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee96b7d-2036-4103-9158-efc66cde3fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.590531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee96b7d-2036-4103-9158-efc66cde3fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.601722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60849a1-eab9-495f-93ae-0b3ee8aaa0f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:26.602455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60849a1-eab9-495f-93ae-0b3ee8aaa0f0 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.602843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60849a1-eab9-495f-93ae-0b3ee8aaa0f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.603277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60849a1-eab9-495f-93ae-0b3ee8aaa0f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.633227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:26.662802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9ee96b7d-2036-4103-9158-efc66cde3fe0 None None] MFA Rules not processed for user `005ddb1d20df40d9897d4fa625fdb9f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:26.665535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9ee96b7d-2036-4103-9158-efc66cde3fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:26.701589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60849a1-eab9-495f-93ae-0b3ee8aaa0f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:26.706808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60849a1-eab9-495f-93ae-0b3ee8aaa0f0 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:26.732462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 800/1619] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:26.744209 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60849a1-eab9-495f-93ae-0b3ee8aaa0f0 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:26.765414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60849a1-eab9-495f-93ae-0b3ee8aaa0f0 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:26.772814 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272d53a-c8eb-4caf-9c1b-679f89650082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:26.773455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272d53a-c8eb-4caf-9c1b-679f89650082 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.773739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272d53a-c8eb-4caf-9c1b-679f89650082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.774016 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272d53a-c8eb-4caf-9c1b-679f89650082 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.774696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272d53a-c8eb-4caf-9c1b-679f89650082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:26.776920 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272d53a-c8eb-4caf-9c1b-679f89650082 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:26.813586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 801/1620] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:26 2022] POST /identity/v3/projects => generated 405 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:26.819060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272d53a-c8eb-4caf-9c1b-679f89650082 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:26.907013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 820/1621] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:26.914318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272d53a-c8eb-4caf-9c1b-679f89650082 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:26.914705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee96b7d-2036-4103-9158-efc66cde3fe0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:26.919597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22c696-9f55-4346-afb4-86c548aaa4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:26.920568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22c696-9f55-4346-afb4-86c548aaa4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.921078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22c696-9f55-4346-afb4-86c548aaa4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.921479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22c696-9f55-4346-afb4-86c548aaa4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.922340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22c696-9f55-4346-afb4-86c548aaa4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:26.924756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22c696-9f55-4346-afb4-86c548aaa4f8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:26.942571 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 802/1622] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:26.943423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee96b7d-2036-4103-9158-efc66cde3fe0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:26.944547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22c696-9f55-4346-afb4-86c548aaa4f8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:26.956768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78d2e0-a8f1-4615-be63-ef3f246c4885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:26.958895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78d2e0-a8f1-4615-be63-ef3f246c4885 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.959080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22c696-9f55-4346-afb4-86c548aaa4f8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:26.959761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78d2e0-a8f1-4615-be63-ef3f246c4885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.960046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78d2e0-a8f1-4615-be63-ef3f246c4885 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.962213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78d2e0-a8f1-4615-be63-ef3f246c4885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:26.965727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725151f-f106-4ada-8976-9913ea1c5ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:26.966477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725151f-f106-4ada-8976-9913ea1c5ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:26.966893 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725151f-f106-4ada-8976-9913ea1c5ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:26.967524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725151f-f106-4ada-8976-9913ea1c5ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:26.968363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725151f-f106-4ada-8976-9913ea1c5ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:26.969726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78d2e0-a8f1-4615-be63-ef3f246c4885 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:26.970859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725151f-f106-4ada-8976-9913ea1c5ebe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:27.014137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 803/1623] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:26 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:27.015444 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725151f-f106-4ada-8976-9913ea1c5ebe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:27.027750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 821/1624] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:26 2022] POST /identity/v3/users => generated 309 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:27.028698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78d2e0-a8f1-4615-be63-ef3f246c4885 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:27.041195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:27.053596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78d2e0-a8f1-4615-be63-ef3f246c4885 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:27.191781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725151f-f106-4ada-8976-9913ea1c5ebe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:27.191781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf7040-1d4c-42b9-917e-657e0154cf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:27.191781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf7040-1d4c-42b9-917e-657e0154cf40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:27.191781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf7040-1d4c-42b9-917e-657e0154cf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:27.193613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf7040-1d4c-42b9-917e-657e0154cf40 admin admin] PATH_INFO: `/v3/projects/64286657ea7c42a8b229ccfcf6d9deca/users/f4040f65194a43169f0fa2d74bc97f1d/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:27.207757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32abfdc-b70c-4f7c-b1a6-2cb01c572caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:27.208406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32abfdc-b70c-4f7c-b1a6-2cb01c572caa admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:27.208687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32abfdc-b70c-4f7c-b1a6-2cb01c572caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:27.211561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32abfdc-b70c-4f7c-b1a6-2cb01c572caa admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:27.212894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32abfdc-b70c-4f7c-b1a6-2cb01c572caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:27.218101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32abfdc-b70c-4f7c-b1a6-2cb01c572caa admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:27.228047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:27.244919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf7040-1d4c-42b9-917e-657e0154cf40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64286657ea7c42a8b229ccfcf6d9deca, user_id=f4040f65194a43169f0fa2d74bc97f1d, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:27.252247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf7040-1d4c-42b9-917e-657e0154cf40 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:27.288415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 822/1625] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:27 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:27.290507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32abfdc-b70c-4f7c-b1a6-2cb01c572caa admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:27.291239 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 804/1626] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:27 2022] PUT /identity/v3/projects/64286657ea7c42a8b229ccfcf6d9deca/users/f4040f65194a43169f0fa2d74bc97f1d/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:27.292765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf7040-1d4c-42b9-917e-657e0154cf40 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:27.317507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32abfdc-b70c-4f7c-b1a6-2cb01c572caa admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:27.425799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf7040-1d4c-42b9-917e-657e0154cf40 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:27.433612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579ef20-fa6d-4221-91e3-9f7d9fe623b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:27.434242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579ef20-fa6d-4221-91e3-9f7d9fe623b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:27.434831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579ef20-fa6d-4221-91e3-9f7d9fe623b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:27.434831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579ef20-fa6d-4221-91e3-9f7d9fe623b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:27.435502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579ef20-fa6d-4221-91e3-9f7d9fe623b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:27.437095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579ef20-fa6d-4221-91e3-9f7d9fe623b1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:27.444832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18908f13-949c-49f7-804d-6ab585b9c99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:27.445336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18908f13-949c-49f7-804d-6ab585b9c99c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:27.445599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18908f13-949c-49f7-804d-6ab585b9c99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:27.445912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18908f13-949c-49f7-804d-6ab585b9c99c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:27.446676 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18908f13-949c-49f7-804d-6ab585b9c99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:27.448396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18908f13-949c-49f7-804d-6ab585b9c99c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:27.463095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 823/1627] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:27.465937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579ef20-fa6d-4221-91e3-9f7d9fe623b1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:27.469333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 805/1628] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:27.470672 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983b10e-7a3b-41e8-a5eb-88714ddbc548 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:27.471021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983b10e-7a3b-41e8-a5eb-88714ddbc548 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:27.471302 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983b10e-7a3b-41e8-a5eb-88714ddbc548 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:27.477627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 806/1629] 10.222.0.46 () {54 vars in 912 bytes} [Thu Jul 28 03:59:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:59:27.478481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983b10e-7a3b-41e8-a5eb-88714ddbc548 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:27.485656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579ef20-fa6d-4221-91e3-9f7d9fe623b1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:27.494130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc217c-322e-4f0d-82bf-0b817c74aa76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:27.494761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc217c-322e-4f0d-82bf-0b817c74aa76 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:27.495089 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc217c-322e-4f0d-82bf-0b817c74aa76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:27.495363 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc217c-322e-4f0d-82bf-0b817c74aa76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:27.496698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc217c-322e-4f0d-82bf-0b817c74aa76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:27.498664 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc217c-322e-4f0d-82bf-0b817c74aa76 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:27.507244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983b10e-7a3b-41e8-a5eb-88714ddbc548 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:27.510635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 824/1630] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:27.511511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc217c-322e-4f0d-82bf-0b817c74aa76 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:27.524895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc217c-322e-4f0d-82bf-0b817c74aa76 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:27.525494 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973012b-9e83-4c07-8c82-ba1cdf5f693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:27.526598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973012b-9e83-4c07-8c82-ba1cdf5f693c admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:27.527214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973012b-9e83-4c07-8c82-ba1cdf5f693c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:27.527655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973012b-9e83-4c07-8c82-ba1cdf5f693c admin admin] PATH_INFO: `/v3/projects/48f7c63acc184def9ff0c2dcd6dd3793/users/04b148623a15466088befb1fd2adcf09/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:27.530475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ac19e-047c-42f8-b3cf-3f25a1c72d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:27.536417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ac19e-047c-42f8-b3cf-3f25a1c72d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:27.536676 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ac19e-047c-42f8-b3cf-3f25a1c72d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:27.536974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ac19e-047c-42f8-b3cf-3f25a1c72d68 admin admin] PATH_INFO: `/v3/projects/aff9b02e532e4298abbfadc60a4122f7/users/97f1a9a2217047c6bc95b5ba5ea4af3e/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:27.560497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ac19e-047c-42f8-b3cf-3f25a1c72d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff9b02e532e4298abbfadc60a4122f7, user_id=97f1a9a2217047c6bc95b5ba5ea4af3e, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:27.563228 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ac19e-047c-42f8-b3cf-3f25a1c72d68 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:27.589584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 825/1631] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:27 2022] PUT /identity/v3/projects/aff9b02e532e4298abbfadc60a4122f7/users/97f1a9a2217047c6bc95b5ba5ea4af3e/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:27.591110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e5848-7650-4097-b773-1c3ccc5a7fb9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:27.591405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e5848-7650-4097-b773-1c3ccc5a7fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:27.591680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e5848-7650-4097-b773-1c3ccc5a7fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:27.600658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973012b-9e83-4c07-8c82-ba1cdf5f693c admin admin] RBAC: Authorizing `identity:create_grant(project_id=48f7c63acc184def9ff0c2dcd6dd3793, user_id=04b148623a15466088befb1fd2adcf09, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:27.613294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973012b-9e83-4c07-8c82-ba1cdf5f693c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:27.636114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fc0e5848-7650-4097-b773-1c3ccc5a7fb9 None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:27.637473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fc0e5848-7650-4097-b773-1c3ccc5a7fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:27.669567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 807/1632] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:27 2022] PUT /identity/v3/projects/48f7c63acc184def9ff0c2dcd6dd3793/users/04b148623a15466088befb1fd2adcf09/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:27.670713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973012b-9e83-4c07-8c82-ba1cdf5f693c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:27.753639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:27.788632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 826/1633] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 03:59:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:27.789796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e5848-7650-4097-b773-1c3ccc5a7fb9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:27.843960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973012b-9e83-4c07-8c82-ba1cdf5f693c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:27.851725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a18418-c04e-484e-820b-910b97069eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:27.852476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a18418-c04e-484e-820b-910b97069eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:27.854251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a18418-c04e-484e-820b-910b97069eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:27.858106 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a18418-c04e-484e-820b-910b97069eab admin admin] PATH_INFO: `/v3/projects/64286657ea7c42a8b229ccfcf6d9deca/users/f4040f65194a43169f0fa2d74bc97f1d/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:27.864887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a18418-c04e-484e-820b-910b97069eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=64286657ea7c42a8b229ccfcf6d9deca, user_id=f4040f65194a43169f0fa2d74bc97f1d, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:27.868968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a18418-c04e-484e-820b-910b97069eab admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:27.899966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 808/1634] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:27 2022] PUT /identity/v3/projects/64286657ea7c42a8b229ccfcf6d9deca/users/f4040f65194a43169f0fa2d74bc97f1d/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:27.901136 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a18418-c04e-484e-820b-910b97069eab admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:27.941401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e5848-7650-4097-b773-1c3ccc5a7fb9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:28.071662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:28.076556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a18418-c04e-484e-820b-910b97069eab admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:28.081136 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8aaeb-5aea-4be8-9cd4-9462c0da0c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:28.081975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8aaeb-5aea-4be8-9cd4-9462c0da0c33 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:28.082322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8aaeb-5aea-4be8-9cd4-9462c0da0c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:28.082604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8aaeb-5aea-4be8-9cd4-9462c0da0c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:28.083868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03475dde-0e67-47cb-9079-de8b6be67840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:28.084270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8aaeb-5aea-4be8-9cd4-9462c0da0c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:28.085033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03475dde-0e67-47cb-9079-de8b6be67840 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:28.085509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03475dde-0e67-47cb-9079-de8b6be67840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:28.086072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8aaeb-5aea-4be8-9cd4-9462c0da0c33 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:28.086260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03475dde-0e67-47cb-9079-de8b6be67840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:28.087222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03475dde-0e67-47cb-9079-de8b6be67840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:28.089450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03475dde-0e67-47cb-9079-de8b6be67840 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:28.125241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 827/1635] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:28.126147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8aaeb-5aea-4be8-9cd4-9462c0da0c33 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:28.132076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 809/1636] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:28.137078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdce73-b77f-4fd8-9284-222e6c28972f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:28.137484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdce73-b77f-4fd8-9284-222e6c28972f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:28.139007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdce73-b77f-4fd8-9284-222e6c28972f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:28.202976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bbbdce73-b77f-4fd8-9284-222e6c28972f None None] MFA Rules not processed for user `f4040f65194a43169f0fa2d74bc97f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:28.204948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bbbdce73-b77f-4fd8-9284-222e6c28972f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:28.240409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8aaeb-5aea-4be8-9cd4-9462c0da0c33 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:28.246437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e5a71-2e27-4f81-a518-75819904159d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:28.248671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e5a71-2e27-4f81-a518-75819904159d service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:28.249627 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e5a71-2e27-4f81-a518-75819904159d service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:28.249627 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e5a71-2e27-4f81-a518-75819904159d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:28.362594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:28.390148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 810/1637] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:28 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:28.391156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdce73-b77f-4fd8-9284-222e6c28972f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:28.427096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdce73-b77f-4fd8-9284-222e6c28972f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:28.432020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e5a71-2e27-4f81-a518-75819904159d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:28.435053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e5a71-2e27-4f81-a518-75819904159d service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:28.445702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334112b-05ec-45df-b2d6-d6008ab95a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:28.447520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334112b-05ec-45df-b2d6-d6008ab95a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:28.448164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334112b-05ec-45df-b2d6-d6008ab95a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:28.448650 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334112b-05ec-45df-b2d6-d6008ab95a55 admin admin] PATH_INFO: `/v3/projects/aff9b02e532e4298abbfadc60a4122f7/users/97f1a9a2217047c6bc95b5ba5ea4af3e/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:28.450503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 828/1638] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:28.451551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e5a71-2e27-4f81-a518-75819904159d service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:28.457474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334112b-05ec-45df-b2d6-d6008ab95a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff9b02e532e4298abbfadc60a4122f7, user_id=97f1a9a2217047c6bc95b5ba5ea4af3e, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:28.464918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334112b-05ec-45df-b2d6-d6008ab95a55 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:28.467578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e5a71-2e27-4f81-a518-75819904159d service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:28.474718 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f966fbe-c17b-4c4b-ac3f-fb83f343e2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:28.475496 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f966fbe-c17b-4c4b-ac3f-fb83f343e2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:28.476556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f966fbe-c17b-4c4b-ac3f-fb83f343e2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:28.476850 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f966fbe-c17b-4c4b-ac3f-fb83f343e2f1 admin admin] PATH_INFO: `/v3/projects/48f7c63acc184def9ff0c2dcd6dd3793/users/04b148623a15466088befb1fd2adcf09/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:28.479712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f966fbe-c17b-4c4b-ac3f-fb83f343e2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48f7c63acc184def9ff0c2dcd6dd3793, user_id=04b148623a15466088befb1fd2adcf09, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:28.482832 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f966fbe-c17b-4c4b-ac3f-fb83f343e2f1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:28.508148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 811/1640] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:28 2022] PUT /identity/v3/projects/aff9b02e532e4298abbfadc60a4122f7/users/97f1a9a2217047c6bc95b5ba5ea4af3e/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:28.508380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 829/1640] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:28 2022] PUT /identity/v3/projects/48f7c63acc184def9ff0c2dcd6dd3793/users/04b148623a15466088befb1fd2adcf09/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:28.509399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334112b-05ec-45df-b2d6-d6008ab95a55 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:28.509952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3d2cd-d516-4b5d-9260-7b4ad3c5cbce None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:28.510665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3d2cd-d516-4b5d-9260-7b4ad3c5cbce None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:28.510665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3d2cd-d516-4b5d-9260-7b4ad3c5cbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:28.557088 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-77d3d2cd-d516-4b5d-9260-7b4ad3c5cbce None None] MFA Rules not processed for user `04b148623a15466088befb1fd2adcf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:28.558635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-77d3d2cd-d516-4b5d-9260-7b4ad3c5cbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:28.640341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334112b-05ec-45df-b2d6-d6008ab95a55 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:28.727192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 830/1641] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:28 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:28.728673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffab35-c88f-44d8-a66e-33dfdb88c05a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:28.728974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffab35-c88f-44d8-a66e-33dfdb88c05a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:28.729259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffab35-c88f-44d8-a66e-33dfdb88c05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:28.775843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fc4cb-d311-49c5-941c-c8aeab481c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:28.777303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fc4cb-d311-49c5-941c-c8aeab481c0c service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:28.779954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fc4cb-d311-49c5-941c-c8aeab481c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:28.780264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fc4cb-d311-49c5-941c-c8aeab481c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:28.794321 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-52ffab35-c88f-44d8-a66e-33dfdb88c05a None None] MFA Rules not processed for user `97f1a9a2217047c6bc95b5ba5ea4af3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:28.795640 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-52ffab35-c88f-44d8-a66e-33dfdb88c05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:28.876265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:28.947936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fc4cb-d311-49c5-941c-c8aeab481c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:28.951523 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fc4cb-d311-49c5-941c-c8aeab481c0c service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:28.966549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 831/1642] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:28.982593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 812/1643] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:29.077841 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffab35-c88f-44d8-a66e-33dfdb88c05a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:29.091638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffab35-c88f-44d8-a66e-33dfdb88c05a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:29.101109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8c5c9-5962-4975-81e5-535e9dc8ac68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:29.101754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8c5c9-5962-4975-81e5-535e9dc8ac68 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:29.102702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8c5c9-5962-4975-81e5-535e9dc8ac68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:29.103033 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8c5c9-5962-4975-81e5-535e9dc8ac68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:29.125857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8c5c9-5962-4975-81e5-535e9dc8ac68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:29.128085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8c5c9-5962-4975-81e5-535e9dc8ac68 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:29.151436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 832/1644] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:29 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:29.261080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fc4cb-d311-49c5-941c-c8aeab481c0c service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:29.269955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fc4cb-d311-49c5-941c-c8aeab481c0c service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:29.280046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981dd92-63ca-45ca-a5bd-1f14bed6b11c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:29.280798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981dd92-63ca-45ca-a5bd-1f14bed6b11c service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:29.281089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981dd92-63ca-45ca-a5bd-1f14bed6b11c service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:29.281396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981dd92-63ca-45ca-a5bd-1f14bed6b11c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:29.308896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981dd92-63ca-45ca-a5bd-1f14bed6b11c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:29.310350 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981dd92-63ca-45ca-a5bd-1f14bed6b11c service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:29.335019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 813/1645] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:29.416638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8c5c9-5962-4975-81e5-535e9dc8ac68 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:29.426182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8c5c9-5962-4975-81e5-535e9dc8ac68 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:29.436260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e0fe7-f511-41b2-a486-53d242ce7025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:29.436914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e0fe7-f511-41b2-a486-53d242ce7025 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:29.437192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e0fe7-f511-41b2-a486-53d242ce7025 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:29.437468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e0fe7-f511-41b2-a486-53d242ce7025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:29.464867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:29.568615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e0fe7-f511-41b2-a486-53d242ce7025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:29.570223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e0fe7-f511-41b2-a486-53d242ce7025 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:29.592484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 833/1646] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:30.193753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981dd92-63ca-45ca-a5bd-1f14bed6b11c service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:30.203465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981dd92-63ca-45ca-a5bd-1f14bed6b11c service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:30.219051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43283c5a-594d-4ea9-bb62-02bebe7b6682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:30.220136 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43283c5a-594d-4ea9-bb62-02bebe7b6682 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:30.223181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43283c5a-594d-4ea9-bb62-02bebe7b6682 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:30.225006 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43283c5a-594d-4ea9-bb62-02bebe7b6682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:30.335870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e0fe7-f511-41b2-a486-53d242ce7025 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:30.443335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43283c5a-594d-4ea9-bb62-02bebe7b6682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:30.448345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43283c5a-594d-4ea9-bb62-02bebe7b6682 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:30.471829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 814/1647] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:30.488051 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e0fe7-f511-41b2-a486-53d242ce7025 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:30.494416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:30.495682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:30.495682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:30.495974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] PATH_INFO: `/v3/users/da9842e3910641e7851c50ad541ea569` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:30.497410 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da9842e3910641e7851c50ad541ea569)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:30.498656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:59:30.498656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 28 03:59:30.498656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: Jul 28 03:59:30.500638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:30.602527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da9842e3910641e7851c50ad541ea569'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:30.603283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da9842e3910641e7851c50ad541ea569'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:30.628762 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da9842e3910641e7851c50ad541ea569'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:30.640360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da9842e3910641e7851c50ad541ea569'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:30.662434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da9842e3910641e7851c50ad541ea569'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:30.685004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 834/1648] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 03:59:30 2022] DELETE /identity/v3/users/da9842e3910641e7851c50ad541ea569 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 03:59:30.690221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ff956-5e00-4b6b-94e7-4ae69657f2d0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:30.690751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ff956-5e00-4b6b-94e7-4ae69657f2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:30.691303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ff956-5e00-4b6b-94e7-4ae69657f2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:30.726831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4b5ff956-5e00-4b6b-94e7-4ae69657f2d0 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:30.728852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4b5ff956-5e00-4b6b-94e7-4ae69657f2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:30.855734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 815/1649] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:30.904773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:30.906817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:31.010228 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:31.054518 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f4717-a26c-4fbd-b2f0-c1168d1c88d3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:31.059451 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae168ba2-2601-49b6-be41-56ff7956adeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:31.060681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae168ba2-2601-49b6-be41-56ff7956adeb service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:31.061130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae168ba2-2601-49b6-be41-56ff7956adeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:31.061535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae168ba2-2601-49b6-be41-56ff7956adeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:31.073707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c53cc-3fb9-4988-9876-7a9b679d8e30 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:31.074113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c53cc-3fb9-4988-9876-7a9b679d8e30 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:31.074562 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c53cc-3fb9-4988-9876-7a9b679d8e30 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:31.076426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 816/1650] 10.222.0.46 () {54 vars in 962 bytes} [Thu Jul 28 03:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:59:31.080942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae168ba2-2601-49b6-be41-56ff7956adeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:31.082622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae168ba2-2601-49b6-be41-56ff7956adeb service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:31.086114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712d76d-cf1a-443d-9678-2ae4009490b4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:31.086114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712d76d-cf1a-443d-9678-2ae4009490b4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:31.086495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712d76d-cf1a-443d-9678-2ae4009490b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:31.112428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 835/1651] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:31.141695 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0712d76d-cf1a-443d-9678-2ae4009490b4 None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:31.143800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0712d76d-cf1a-443d-9678-2ae4009490b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:31.236615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:31.287401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 817/1652] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Jul 28 03:59:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:31.293733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7476b-2e19-4e68-8326-b425909f45f4 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:31.294060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7476b-2e19-4e68-8326-b425909f45f4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:31.294300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7476b-2e19-4e68-8326-b425909f45f4 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:31.296281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 836/1653] 10.222.0.46 () {54 vars in 912 bytes} [Thu Jul 28 03:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:59:31.304981 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cbfae-8a73-43b2-b494-f65db8e1c987 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:31.305691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cbfae-8a73-43b2-b494-f65db8e1c987 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:31.322433 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cbfae-8a73-43b2-b494-f65db8e1c987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:31.355951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7e2cbfae-8a73-43b2-b494-f65db8e1c987 None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:31.358253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7e2cbfae-8a73-43b2-b494-f65db8e1c987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:31.426340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 818/1654] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 03:59:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:31.478147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7476b-2e19-4e68-8326-b425909f45f4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:31.478147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e1d7476b-2e19-4e68-8326-b425909f45f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:31.600996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7476b-2e19-4e68-8326-b425909f45f4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:31.613948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc32f2-9f06-4317-bd6e-8616252ff44d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:31.614759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc32f2-9f06-4317-bd6e-8616252ff44d service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:31.614884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc32f2-9f06-4317-bd6e-8616252ff44d service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:31.615346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc32f2-9f06-4317-bd6e-8616252ff44d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:31.632174 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc32f2-9f06-4317-bd6e-8616252ff44d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:31.633612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc32f2-9f06-4317-bd6e-8616252ff44d service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:31.668349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 837/1655] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:32.000275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cbfae-8a73-43b2-b494-f65db8e1c987 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:32.002703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7e2cbfae-8a73-43b2-b494-f65db8e1c987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:32.034948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cbfae-8a73-43b2-b494-f65db8e1c987 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:32.046346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:32.047886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:32.054136 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:32.054599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] PATH_INFO: `/v3/projects/d04e2d7f19d6459899ccb47d0668b22f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:32.060580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d04e2d7f19d6459899ccb47d0668b22f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:32.064219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 03:59:32.064219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 28 03:59:32.064219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: Jul 28 03:59:32.067084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:32.234708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd04e2d7f19d6459899ccb47d0668b22f', 'request_id': 'req-1a2401e8-abb2-4c7e-a411-9d2937b545a9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:32.247797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd04e2d7f19d6459899ccb47d0668b22f', 'request_id': 'req-1a2401e8-abb2-4c7e-a411-9d2937b545a9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:32.276804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd04e2d7f19d6459899ccb47d0668b22f', 'request_id': 'req-1a2401e8-abb2-4c7e-a411-9d2937b545a9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:32.331770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] The token cache is being invalidate because project d04e2d7f19d6459899ccb47d0668b22f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 03:59:32.333128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:32.337259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:32.340743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 819/1656] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 03:59:31 2022] DELETE /identity/v3/projects/d04e2d7f19d6459899ccb47d0668b22f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:32.344909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc32f2-9f06-4317-bd6e-8616252ff44d service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:32.364016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc32f2-9f06-4317-bd6e-8616252ff44d service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:32.583226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:32.592994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:32.594823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:32.596854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] PATH_INFO: `/v3/users/04b148623a15466088befb1fd2adcf09` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:32.600217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] RBAC: Authorizing `identity:delete_user(user_id=04b148623a15466088befb1fd2adcf09)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:32.606119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:32.778443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04b148623a15466088befb1fd2adcf09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:32.784902 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04b148623a15466088befb1fd2adcf09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:32.819628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04b148623a15466088befb1fd2adcf09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:32.851338 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04b148623a15466088befb1fd2adcf09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:32.861115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:32.877855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04b148623a15466088befb1fd2adcf09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:32.905982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 838/1657] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 03:59:32 2022] DELETE /identity/v3/users/04b148623a15466088befb1fd2adcf09 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:34.764515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:34.766267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:34.800398 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2401e8-abb2-4c7e-a411-9d2937b545a9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:35.021310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:35.022175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:35.023705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:35.024602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] PATH_INFO: `/v3/projects/48f7c63acc184def9ff0c2dcd6dd3793` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:35.026086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48f7c63acc184def9ff0c2dcd6dd3793)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:35.027352 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:35.028888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:35.029540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:35.060651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a00429-5b6e-4fb2-bf00-17591fd5c27c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:35.078120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d6869-48ef-4827-a574-951fc44a4906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:35.081253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d6869-48ef-4827-a574-951fc44a4906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:35.081687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d6869-48ef-4827-a574-951fc44a4906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:35.082153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d6869-48ef-4827-a574-951fc44a4906 admin admin] PATH_INFO: `/v3/users/46300b1878604dabbe36d06ede537258` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:35.085110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d6869-48ef-4827-a574-951fc44a4906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46300b1878604dabbe36d06ede537258)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:35.090823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d6869-48ef-4827-a574-951fc44a4906 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:35.145953 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48f7c63acc184def9ff0c2dcd6dd3793', 'request_id': 'req-7684e135-7944-454d-97c3-b5d44be34ce4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:35.154477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48f7c63acc184def9ff0c2dcd6dd3793', 'request_id': 'req-7684e135-7944-454d-97c3-b5d44be34ce4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:35.165912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48f7c63acc184def9ff0c2dcd6dd3793', 'request_id': 'req-7684e135-7944-454d-97c3-b5d44be34ce4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:35.169759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:35.184878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] The token cache is being invalidate because project 48f7c63acc184def9ff0c2dcd6dd3793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 03:59:35.186071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:35.187554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7684e135-7944-454d-97c3-b5d44be34ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:35.193865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 820/1658] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 03:59:34 2022] DELETE /identity/v3/projects/48f7c63acc184def9ff0c2dcd6dd3793 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:35.197990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad7aea-c1c5-43a1-8e92-6821c2587fb4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:35.199131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad7aea-c1c5-43a1-8e92-6821c2587fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:35.201074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad7aea-c1c5-43a1-8e92-6821c2587fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:35.249201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-366d6869-48ef-4827-a574-951fc44a4906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46300b1878604dabbe36d06ede537258'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:35.250153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-366d6869-48ef-4827-a574-951fc44a4906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46300b1878604dabbe36d06ede537258'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:35.276191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8fad7aea-c1c5-43a1-8e92-6821c2587fb4 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:35.285699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8fad7aea-c1c5-43a1-8e92-6821c2587fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:35.300648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-366d6869-48ef-4827-a574-951fc44a4906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46300b1878604dabbe36d06ede537258'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:35.311212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-366d6869-48ef-4827-a574-951fc44a4906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46300b1878604dabbe36d06ede537258'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:35.332277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-366d6869-48ef-4827-a574-951fc44a4906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46300b1878604dabbe36d06ede537258'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:35.342775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 839/1659] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 03:59:35 2022] DELETE /identity/v3/users/46300b1878604dabbe36d06ede537258 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:35.347914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69178b79-20bd-4025-b541-f2bc493aae52 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:35.348411 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69178b79-20bd-4025-b541-f2bc493aae52 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:35.348821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69178b79-20bd-4025-b541-f2bc493aae52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:35.394016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-69178b79-20bd-4025-b541-f2bc493aae52 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:35.407068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-69178b79-20bd-4025-b541-f2bc493aae52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:35.560428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 821/1660] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:35.562159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc1598-6401-4279-bee6-598bd0edae95 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:35.562535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc1598-6401-4279-bee6-598bd0edae95 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:35.562842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc1598-6401-4279-bee6-598bd0edae95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:35.581411 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 840/1661] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:35.624859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cbfc1598-6401-4279-bee6-598bd0edae95 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:35.628491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cbfc1598-6401-4279-bee6-598bd0edae95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:35.658887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 822/1662] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:35.659706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69178b79-20bd-4025-b541-f2bc493aae52 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:35.672297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69178b79-20bd-4025-b541-f2bc493aae52 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:35.692687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6dac8-1d68-4b33-ba6f-c0058f492628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:35.693571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6dac8-1d68-4b33-ba6f-c0058f492628 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:35.693989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6dac8-1d68-4b33-ba6f-c0058f492628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:35.694396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6dac8-1d68-4b33-ba6f-c0058f492628 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:35.695328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6dac8-1d68-4b33-ba6f-c0058f492628 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:35.695764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6dac8-1d68-4b33-ba6f-c0058f492628 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:35.697517 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6dac8-1d68-4b33-ba6f-c0058f492628 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:35.727793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc1598-6401-4279-bee6-598bd0edae95 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:35.729255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cbfc1598-6401-4279-bee6-598bd0edae95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:35.730152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 841/1663] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:35.734385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6dac8-1d68-4b33-ba6f-c0058f492628 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:35.761060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6dac8-1d68-4b33-ba6f-c0058f492628 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:35.771836 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2de53f-30f7-48e3-8625-29e2ae27905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:35.772638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2de53f-30f7-48e3-8625-29e2ae27905b admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:35.772742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2de53f-30f7-48e3-8625-29e2ae27905b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:35.772998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2de53f-30f7-48e3-8625-29e2ae27905b admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:35.773669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2de53f-30f7-48e3-8625-29e2ae27905b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:35.775811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2de53f-30f7-48e3-8625-29e2ae27905b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:35.849158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 842/1664] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:35 2022] POST /identity/v3/projects => generated 355 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:59:35.850276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2de53f-30f7-48e3-8625-29e2ae27905b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:35.909915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc1598-6401-4279-bee6-598bd0edae95 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:35.936656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:36.036542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2de53f-30f7-48e3-8625-29e2ae27905b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:36.045282 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562909bf-ddac-46fb-af41-3dfa2e54c794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:36.046058 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562909bf-ddac-46fb-af41-3dfa2e54c794 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:36.046453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562909bf-ddac-46fb-af41-3dfa2e54c794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:36.046811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562909bf-ddac-46fb-af41-3dfa2e54c794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:36.062221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab1a41-c798-4dd8-ae7c-61fb4ff00cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:36.063261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab1a41-c798-4dd8-ae7c-61fb4ff00cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:36.063680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab1a41-c798-4dd8-ae7c-61fb4ff00cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:36.064086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab1a41-c798-4dd8-ae7c-61fb4ff00cb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:36.065780 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab1a41-c798-4dd8-ae7c-61fb4ff00cb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:36.068235 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab1a41-c798-4dd8-ae7c-61fb4ff00cb3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:36.088617 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562909bf-ddac-46fb-af41-3dfa2e54c794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:36.090097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562909bf-ddac-46fb-af41-3dfa2e54c794 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:36.108809 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 823/1665] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:36.113713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484053eb-2b7a-4dbb-89c1-778fbb78a02d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:36.114007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484053eb-2b7a-4dbb-89c1-778fbb78a02d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:36.114268 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484053eb-2b7a-4dbb-89c1-778fbb78a02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:36.125402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 843/1666] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:35 2022] POST /identity/v3/users => generated 278 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:36.131325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab1a41-c798-4dd8-ae7c-61fb4ff00cb3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:36.149027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab1a41-c798-4dd8-ae7c-61fb4ff00cb3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:36.168956 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66590f4b-9b54-4948-a6cf-2f0d1b23388e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:36.169935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66590f4b-9b54-4948-a6cf-2f0d1b23388e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:36.170387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66590f4b-9b54-4948-a6cf-2f0d1b23388e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:36.170916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66590f4b-9b54-4948-a6cf-2f0d1b23388e admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:36.171932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66590f4b-9b54-4948-a6cf-2f0d1b23388e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:36.174175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66590f4b-9b54-4948-a6cf-2f0d1b23388e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:36.201797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 844/1667] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 03:59:36.206681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66590f4b-9b54-4948-a6cf-2f0d1b23388e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:36.225382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66590f4b-9b54-4948-a6cf-2f0d1b23388e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:36.243501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbad2e5-a3ad-4f26-a44d-5175889d993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:36.244150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbad2e5-a3ad-4f26-a44d-5175889d993d admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:36.244430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbad2e5-a3ad-4f26-a44d-5175889d993d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:36.244707 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbad2e5-a3ad-4f26-a44d-5175889d993d admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:36.245404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbad2e5-a3ad-4f26-a44d-5175889d993d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:36.245814 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-484053eb-2b7a-4dbb-89c1-778fbb78a02d None None] MFA Rules not processed for user `c3308761e9d2495e95e127bbc7b9c5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:36.247122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbad2e5-a3ad-4f26-a44d-5175889d993d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:36.250891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-484053eb-2b7a-4dbb-89c1-778fbb78a02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:36.287783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 845/1668] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:36.296068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbad2e5-a3ad-4f26-a44d-5175889d993d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:36.319621 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbad2e5-a3ad-4f26-a44d-5175889d993d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:36.328002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18abb08-524a-4861-bfea-85ac9e8787d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:36.331196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18abb08-524a-4861-bfea-85ac9e8787d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:36.331917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18abb08-524a-4861-bfea-85ac9e8787d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:36.332741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18abb08-524a-4861-bfea-85ac9e8787d9 admin admin] PATH_INFO: `/v3/projects/f80df879907e4c94b2b55431a434417d/users/0609476b0b494fc69dd159b7112136cc/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:36.360064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:36.365422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18abb08-524a-4861-bfea-85ac9e8787d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f80df879907e4c94b2b55431a434417d, user_id=0609476b0b494fc69dd159b7112136cc, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:36.368449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18abb08-524a-4861-bfea-85ac9e8787d9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:36.395468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 846/1669] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:36 2022] PUT /identity/v3/projects/f80df879907e4c94b2b55431a434417d/users/0609476b0b494fc69dd159b7112136cc/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:36.408873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6a4e8-7d3c-4ada-802c-7e5661087c95 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:36.409400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6a4e8-7d3c-4ada-802c-7e5661087c95 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:36.409867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6a4e8-7d3c-4ada-802c-7e5661087c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:36.436090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 824/1670] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:36.442713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484053eb-2b7a-4dbb-89c1-778fbb78a02d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:36.445161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-484053eb-2b7a-4dbb-89c1-778fbb78a02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:36.478090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-89a6a4e8-7d3c-4ada-802c-7e5661087c95 None None] MFA Rules not processed for user `0609476b0b494fc69dd159b7112136cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:36.481125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-89a6a4e8-7d3c-4ada-802c-7e5661087c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:36.597712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 847/1671] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:36 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:36.604593 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6a4e8-7d3c-4ada-802c-7e5661087c95 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:36.628689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484053eb-2b7a-4dbb-89c1-778fbb78a02d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:36.763606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6a4e8-7d3c-4ada-802c-7e5661087c95 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:36.776930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee5b37-e964-433c-aed8-d8df5f97bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:36.777534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14534f4-3f87-4047-894e-dbc4240fc782 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:36.778135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee5b37-e964-433c-aed8-d8df5f97bf6b admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:36.779237 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee5b37-e964-433c-aed8-d8df5f97bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:36.779599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14534f4-3f87-4047-894e-dbc4240fc782 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:36.780124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee5b37-e964-433c-aed8-d8df5f97bf6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:36.784834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14534f4-3f87-4047-894e-dbc4240fc782 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:36.785870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee5b37-e964-433c-aed8-d8df5f97bf6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:36.786593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14534f4-3f87-4047-894e-dbc4240fc782 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:36.793237 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee5b37-e964-433c-aed8-d8df5f97bf6b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:36.841048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:36.875358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 848/1672] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:36 2022] POST /identity/v3/projects => generated 357 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:36.878704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee5b37-e964-433c-aed8-d8df5f97bf6b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:36.886334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3eee5b37-e964-433c-aed8-d8df5f97bf6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:36.944307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14534f4-3f87-4047-894e-dbc4240fc782 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:36.950516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14534f4-3f87-4047-894e-dbc4240fc782 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:36.971526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 825/1673] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 03:59:36.973383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14534f4-3f87-4047-894e-dbc4240fc782 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:37.020408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14534f4-3f87-4047-894e-dbc4240fc782 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:37.043401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:37.044976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee5b37-e964-433c-aed8-d8df5f97bf6b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:37.062612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:37.063313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:37.063596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:37.063872 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] PATH_INFO: `/v3/projects/6cac357ba49f45b8b4305db5fe1b5aa9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:37.072961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cac357ba49f45b8b4305db5fe1b5aa9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:37.075007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:37.135421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b6cdd-647c-406a-9245-52c84b514f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:37.137036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b6cdd-647c-406a-9245-52c84b514f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:37.137530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b6cdd-647c-406a-9245-52c84b514f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:37.138144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b6cdd-647c-406a-9245-52c84b514f93 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:37.139078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b6cdd-647c-406a-9245-52c84b514f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:37.144511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b6cdd-647c-406a-9245-52c84b514f93 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:37.220023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 826/1674] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:36 2022] POST /identity/v3/users => generated 285 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:37.225849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b6cdd-647c-406a-9245-52c84b514f93 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:37.256507 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b6cdd-647c-406a-9245-52c84b514f93 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:37.262153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cac357ba49f45b8b4305db5fe1b5aa9', 'request_id': 'req-f94d862b-d659-43c0-8b11-ba17a72b47c2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:37.279769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cac357ba49f45b8b4305db5fe1b5aa9', 'request_id': 'req-f94d862b-d659-43c0-8b11-ba17a72b47c2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:37.287744 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cac357ba49f45b8b4305db5fe1b5aa9', 'request_id': 'req-f94d862b-d659-43c0-8b11-ba17a72b47c2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:37.337114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] The token cache is being invalidate because project 6cac357ba49f45b8b4305db5fe1b5aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 03:59:37.337602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:37.340113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:37.346623 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 849/1675] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 03:59:36 2022] DELETE /identity/v3/projects/6cac357ba49f45b8b4305db5fe1b5aa9 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:37.347756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:37.349211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:37.454712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514623a-8dea-412c-bbe2-428c6b6a0578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:37.455494 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514623a-8dea-412c-bbe2-428c6b6a0578 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:37.455762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514623a-8dea-412c-bbe2-428c6b6a0578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:37.456080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514623a-8dea-412c-bbe2-428c6b6a0578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:37.456829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514623a-8dea-412c-bbe2-428c6b6a0578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:37.458497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514623a-8dea-412c-bbe2-428c6b6a0578 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:37.491970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 827/1676] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:37.496282 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514623a-8dea-412c-bbe2-428c6b6a0578 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:37.497933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3514623a-8dea-412c-bbe2-428c6b6a0578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:37.541460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514623a-8dea-412c-bbe2-428c6b6a0578 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:37.566234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14663ff2-c4fc-42fe-8511-ff501b2e0d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:37.567235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14663ff2-c4fc-42fe-8511-ff501b2e0d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:37.567651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14663ff2-c4fc-42fe-8511-ff501b2e0d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:37.568062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14663ff2-c4fc-42fe-8511-ff501b2e0d5b admin admin] PATH_INFO: `/v3/projects/6cf630b7ba0b43bfaad2647f706fbf17/users/f751b14f54224be492c205f228394365/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:37.569570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d862b-d659-43c0-8b11-ba17a72b47c2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:37.585884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:37.586935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:37.587348 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:37.587756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] PATH_INFO: `/v3/users/f4040f65194a43169f0fa2d74bc97f1d` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:37.590084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4040f65194a43169f0fa2d74bc97f1d)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:37.592056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:37.621504 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14663ff2-c4fc-42fe-8511-ff501b2e0d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cf630b7ba0b43bfaad2647f706fbf17, user_id=f751b14f54224be492c205f228394365, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:37.623761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14663ff2-c4fc-42fe-8511-ff501b2e0d5b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:37.655537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 828/1677] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:37 2022] PUT /identity/v3/projects/6cf630b7ba0b43bfaad2647f706fbf17/users/f751b14f54224be492c205f228394365/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 03:59:37.657462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:37.659633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14663ff2-c4fc-42fe-8511-ff501b2e0d5b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:37.723149 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4040f65194a43169f0fa2d74bc97f1d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:37.726234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4040f65194a43169f0fa2d74bc97f1d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:37.752747 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4040f65194a43169f0fa2d74bc97f1d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:37.769439 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4040f65194a43169f0fa2d74bc97f1d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:37.793618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4040f65194a43169f0fa2d74bc97f1d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:37.814059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 850/1678] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 03:59:37 2022] DELETE /identity/v3/users/f4040f65194a43169f0fa2d74bc97f1d => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:37.822185 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14663ff2-c4fc-42fe-8511-ff501b2e0d5b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:37.872314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:37.968902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f9c72-fc59-4092-ac45-1b3d42de22d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:37.969515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9c72-fc59-4092-ac45-1b3d42de22d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:37.969613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9c72-fc59-4092-ac45-1b3d42de22d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:37.969828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9c72-fc59-4092-ac45-1b3d42de22d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:37.975705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f9c72-fc59-4092-ac45-1b3d42de22d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:37.977858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f9c72-fc59-4092-ac45-1b3d42de22d9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:38.002698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 829/1679] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:38.005202 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:38.006869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:38.031200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84ea0c-f1aa-4320-aaa2-7b13ec114fa5 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:38.043097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76b615-0152-428f-9986-182778ca2a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:38.043971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76b615-0152-428f-9986-182778ca2a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:38.044331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76b615-0152-428f-9986-182778ca2a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:38.045745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76b615-0152-428f-9986-182778ca2a30 admin admin] PATH_INFO: `/v3/projects/6cf630b7ba0b43bfaad2647f706fbf17/users/f751b14f54224be492c205f228394365/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:38.050188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76b615-0152-428f-9986-182778ca2a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cf630b7ba0b43bfaad2647f706fbf17, user_id=f751b14f54224be492c205f228394365, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:38.057272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76b615-0152-428f-9986-182778ca2a30 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:38.094070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9958a-6693-4c0b-ab3e-8f8232513c2e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:38.094411 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 851/1680] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:38 2022] PUT /identity/v3/projects/6cf630b7ba0b43bfaad2647f706fbf17/users/f751b14f54224be492c205f228394365/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:38.094501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9958a-6693-4c0b-ab3e-8f8232513c2e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:38.094825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9958a-6693-4c0b-ab3e-8f8232513c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:38.135190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0bf9958a-6693-4c0b-ab3e-8f8232513c2e None None] MFA Rules not processed for user `f751b14f54224be492c205f228394365`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:38.136538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0bf9958a-6693-4c0b-ab3e-8f8232513c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:38.234481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 830/1681] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:38 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:38.243505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76b615-0152-428f-9986-182778ca2a30 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:38.245542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6d76b615-0152-428f-9986-182778ca2a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:38.372856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76b615-0152-428f-9986-182778ca2a30 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:38.383023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ba411-f13a-40b2-84eb-750b43baaa30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:38.383726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ba411-f13a-40b2-84eb-750b43baaa30 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:38.384010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ba411-f13a-40b2-84eb-750b43baaa30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:38.384292 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ba411-f13a-40b2-84eb-750b43baaa30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:38.404597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ba411-f13a-40b2-84eb-750b43baaa30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:38.406057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ba411-f13a-40b2-84eb-750b43baaa30 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:38.433083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 852/1682] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:38 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:38.612827 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9958a-6693-4c0b-ab3e-8f8232513c2e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:38.630366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9958a-6693-4c0b-ab3e-8f8232513c2e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:38.642146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff9517-b5f6-47e9-abca-f9abf3b6407c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:38.644409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff9517-b5f6-47e9-abca-f9abf3b6407c service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:38.645926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff9517-b5f6-47e9-abca-f9abf3b6407c service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:38.646544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff9517-b5f6-47e9-abca-f9abf3b6407c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:38.653038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-22ff9517-b5f6-47e9-abca-f9abf3b6407c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:38.819477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff9517-b5f6-47e9-abca-f9abf3b6407c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:38.822895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff9517-b5f6-47e9-abca-f9abf3b6407c service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:38.849771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 831/1683] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:38 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:39.270106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcd51c-adc2-42e2-bad2-424e8d39ddce None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:39.270564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcd51c-adc2-42e2-bad2-424e8d39ddce None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:39.270657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcd51c-adc2-42e2-bad2-424e8d39ddce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:39.360151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b1bcd51c-adc2-42e2-bad2-424e8d39ddce None None] MFA Rules not processed for user `d09411d75019428a922e12e02c72f4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:39.362037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b1bcd51c-adc2-42e2-bad2-424e8d39ddce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:39.539705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 853/1684] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:39.546210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcec43-ecff-4674-9894-f6a48ebf6f7a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:39.546842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcec43-ecff-4674-9894-f6a48ebf6f7a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:39.546917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcec43-ecff-4674-9894-f6a48ebf6f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:39.585805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcd51c-adc2-42e2-bad2-424e8d39ddce None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:39.587417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b1bcd51c-adc2-42e2-bad2-424e8d39ddce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:39.616609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3dfcec43-ecff-4674-9894-f6a48ebf6f7a None None] MFA Rules not processed for user `d09411d75019428a922e12e02c72f4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:39.621983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3dfcec43-ecff-4674-9894-f6a48ebf6f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:39.630351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:39.664063 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 832/1685] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:39.684230 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcec43-ecff-4674-9894-f6a48ebf6f7a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:39.686260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3dfcec43-ecff-4674-9894-f6a48ebf6f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:39.732083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcd51c-adc2-42e2-bad2-424e8d39ddce None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:39.737376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:39.737940 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:39.738225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:39.738499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] PATH_INFO: `/v3/projects/64286657ea7c42a8b229ccfcf6d9deca` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:39.745330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64286657ea7c42a8b229ccfcf6d9deca)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:39.747360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:39.849358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64286657ea7c42a8b229ccfcf6d9deca', 'request_id': 'req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:39.854810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:39.859646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64286657ea7c42a8b229ccfcf6d9deca', 'request_id': 'req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:39.873291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64286657ea7c42a8b229ccfcf6d9deca', 'request_id': 'req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:39.898418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] The token cache is being invalidate because project 64286657ea7c42a8b229ccfcf6d9deca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 03:59:39.898973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:39.903247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5765f8d3-c470-4c1f-a9ff-bafbd61f1be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:39.906150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 854/1686] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 03:59:39 2022] DELETE /identity/v3/projects/64286657ea7c42a8b229ccfcf6d9deca => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 03:59:39.911123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279516a1-3033-44e4-b75d-4fdf062269ab None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:39.911701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279516a1-3033-44e4-b75d-4fdf062269ab None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:39.912041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279516a1-3033-44e4-b75d-4fdf062269ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:39.944179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcec43-ecff-4674-9894-f6a48ebf6f7a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:39.950949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-279516a1-3033-44e4-b75d-4fdf062269ab None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:39.953119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-279516a1-3033-44e4-b75d-4fdf062269ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:40.099458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 855/1687] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:40.108470 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b885824-3b01-48a7-8b0d-5d69b4304b7b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:40.108905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b885824-3b01-48a7-8b0d-5d69b4304b7b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:40.109319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b885824-3b01-48a7-8b0d-5d69b4304b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:40.159861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3ab25-ed17-47d3-ae0e-93d49eb57e0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:40.160521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3ab25-ed17-47d3-ae0e-93d49eb57e0f service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:40.160811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3ab25-ed17-47d3-ae0e-93d49eb57e0f service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:40.161087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3ab25-ed17-47d3-ae0e-93d49eb57e0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:40.167846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7cc3ab25-ed17-47d3-ae0e-93d49eb57e0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:40.167981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2b885824-3b01-48a7-8b0d-5d69b4304b7b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:40.169187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2b885824-3b01-48a7-8b0d-5d69b4304b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:40.193773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 856/1688] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:40.194990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b885824-3b01-48a7-8b0d-5d69b4304b7b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:40.209377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b885824-3b01-48a7-8b0d-5d69b4304b7b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:40.222391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a832db-15c2-42b6-b33e-f523be5e10af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:40.223021 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a832db-15c2-42b6-b33e-f523be5e10af admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:40.223377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a832db-15c2-42b6-b33e-f523be5e10af admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:40.223549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a832db-15c2-42b6-b33e-f523be5e10af admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:40.224271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a832db-15c2-42b6-b33e-f523be5e10af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:40.224654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a832db-15c2-42b6-b33e-f523be5e10af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:40.226453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a832db-15c2-42b6-b33e-f523be5e10af admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:40.260865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 857/1689] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:40.261827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a832db-15c2-42b6-b33e-f523be5e10af admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:40.286022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a832db-15c2-42b6-b33e-f523be5e10af admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:40.315189 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baba16-d00a-46ef-85ab-b8de59882f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:40.316001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baba16-d00a-46ef-85ab-b8de59882f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:40.316284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baba16-d00a-46ef-85ab-b8de59882f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:40.316556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baba16-d00a-46ef-85ab-b8de59882f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:40.317219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6baba16-d00a-46ef-85ab-b8de59882f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:40.319217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6baba16-d00a-46ef-85ab-b8de59882f5b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:40.327807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3ab25-ed17-47d3-ae0e-93d49eb57e0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:40.330349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3ab25-ed17-47d3-ae0e-93d49eb57e0f service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:40.356240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 833/1690] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:40.397648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 858/1691] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:40 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:40.399438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3ab25-ed17-47d3-ae0e-93d49eb57e0f service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:40.426128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3ab25-ed17-47d3-ae0e-93d49eb57e0f service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:40.576937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328afb1c-7bbf-4929-a939-7571e8170593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:40.577730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328afb1c-7bbf-4929-a939-7571e8170593 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:40.578017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328afb1c-7bbf-4929-a939-7571e8170593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:40.578431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328afb1c-7bbf-4929-a939-7571e8170593 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:40.584842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328afb1c-7bbf-4929-a939-7571e8170593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:40.587021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328afb1c-7bbf-4929-a939-7571e8170593 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:40.630636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:40.637799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 834/1692] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:40 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:40.641905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baba16-d00a-46ef-85ab-b8de59882f5b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:40.661281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baba16-d00a-46ef-85ab-b8de59882f5b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:40.684527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213f95a-d3d7-49bd-923f-03e216dabf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:40.685878 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f95a-d3d7-49bd-923f-03e216dabf35 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:40.686372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f95a-d3d7-49bd-923f-03e216dabf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:40.687127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f95a-d3d7-49bd-923f-03e216dabf35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:40.688504 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213f95a-d3d7-49bd-923f-03e216dabf35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:40.690802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213f95a-d3d7-49bd-923f-03e216dabf35 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:40.700907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:40.759188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 859/1693] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 03:59:40.760559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328afb1c-7bbf-4929-a939-7571e8170593 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:40.777070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328afb1c-7bbf-4929-a939-7571e8170593 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:40.803002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4abca-97a2-42c1-837e-6a52a051726c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:40.804727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4abca-97a2-42c1-837e-6a52a051726c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:40.805101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4abca-97a2-42c1-837e-6a52a051726c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:40.805498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4abca-97a2-42c1-837e-6a52a051726c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:40.806363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4abca-97a2-42c1-837e-6a52a051726c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:40.811898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4abca-97a2-42c1-837e-6a52a051726c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:40.858984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 835/1694] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:40.865151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213f95a-d3d7-49bd-923f-03e216dabf35 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:40.882467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213f95a-d3d7-49bd-923f-03e216dabf35 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:40.908435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192297a-ddb5-4aed-a0d5-48ca66a8c147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:40.909308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192297a-ddb5-4aed-a0d5-48ca66a8c147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:40.909602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192297a-ddb5-4aed-a0d5-48ca66a8c147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:40.909997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192297a-ddb5-4aed-a0d5-48ca66a8c147 admin admin] PATH_INFO: `/v3/projects/017b0d1b30a44f5887b0a23e7db3a489/users/f7caf8adb43340e0a5521863113a3daa/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:40.954307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192297a-ddb5-4aed-a0d5-48ca66a8c147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017b0d1b30a44f5887b0a23e7db3a489, user_id=f7caf8adb43340e0a5521863113a3daa, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:40.959016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192297a-ddb5-4aed-a0d5-48ca66a8c147 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:40.986851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 860/1695] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:40 2022] PUT /identity/v3/projects/017b0d1b30a44f5887b0a23e7db3a489/users/f7caf8adb43340e0a5521863113a3daa/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:40.997186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d13aa-8505-4370-8da9-10972b53bdc3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:40.998145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d13aa-8505-4370-8da9-10972b53bdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:40.999266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d13aa-8505-4370-8da9-10972b53bdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:41.085513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-712d13aa-8505-4370-8da9-10972b53bdc3 None None] MFA Rules not processed for user `f7caf8adb43340e0a5521863113a3daa`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:41.087027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-712d13aa-8505-4370-8da9-10972b53bdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:41.253400 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 836/1696] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:41.255475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192297a-ddb5-4aed-a0d5-48ca66a8c147 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:41.409311 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192297a-ddb5-4aed-a0d5-48ca66a8c147 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:41.437802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf74c10-9656-4211-83d9-e109340a9c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:41.439038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf74c10-9656-4211-83d9-e109340a9c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:41.439603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf74c10-9656-4211-83d9-e109340a9c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:41.440468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf74c10-9656-4211-83d9-e109340a9c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:41.441333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf74c10-9656-4211-83d9-e109340a9c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:41.444580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf74c10-9656-4211-83d9-e109340a9c0d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:41.503653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 861/1697] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:41 2022] POST /identity/v3/projects => generated 383 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:41.507139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d13aa-8505-4370-8da9-10972b53bdc3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:41.540574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d13aa-8505-4370-8da9-10972b53bdc3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:41.778207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c11f0a-21e1-4b04-b283-55dbfbcb3dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:41.779127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11f0a-21e1-4b04-b283-55dbfbcb3dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:41.779255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11f0a-21e1-4b04-b283-55dbfbcb3dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:41.779442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11f0a-21e1-4b04-b283-55dbfbcb3dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:41.780183 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c11f0a-21e1-4b04-b283-55dbfbcb3dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:41.782131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c11f0a-21e1-4b04-b283-55dbfbcb3dd3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:41.856500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 837/1698] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:41 2022] POST /identity/v3/users => generated 298 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:59:41.861823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf74c10-9656-4211-83d9-e109340a9c0d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:41.898811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf74c10-9656-4211-83d9-e109340a9c0d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:41.907401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c11f0a-21e1-4b04-b283-55dbfbcb3dd3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:41.909552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c2c11f0a-21e1-4b04-b283-55dbfbcb3dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:41.930156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550e849-59ec-4f88-9f19-1cf554ff3412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:41.933223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550e849-59ec-4f88-9f19-1cf554ff3412 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:41.933830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550e849-59ec-4f88-9f19-1cf554ff3412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:41.935400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550e849-59ec-4f88-9f19-1cf554ff3412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:41.936282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550e849-59ec-4f88-9f19-1cf554ff3412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:41.948890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c11f0a-21e1-4b04-b283-55dbfbcb3dd3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:41.951414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550e849-59ec-4f88-9f19-1cf554ff3412 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:41.986160 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:41.989681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:41.989970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:41.990248 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] PATH_INFO: `/v3/users/4dab551c284242df98e62efdc5de8de7` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:42.000879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dab551c284242df98e62efdc5de8de7)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:42.000879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:42.016473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 862/1699] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:42.018364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea161f2d-4231-4268-a8bd-a59f91a9d919 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:42.018651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea161f2d-4231-4268-a8bd-a59f91a9d919 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:42.023985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea161f2d-4231-4268-a8bd-a59f91a9d919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:42.092313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ea161f2d-4231-4268-a8bd-a59f91a9d919 None None] MFA Rules not processed for user `c3308761e9d2495e95e127bbc7b9c5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:42.093570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ea161f2d-4231-4268-a8bd-a59f91a9d919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:42.163652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dab551c284242df98e62efdc5de8de7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:42.167932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dab551c284242df98e62efdc5de8de7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:42.200360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dab551c284242df98e62efdc5de8de7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:42.209923 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:42.239434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dab551c284242df98e62efdc5de8de7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:42.264283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dab551c284242df98e62efdc5de8de7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:42.285366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 838/1700] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 03:59:41 2022] DELETE /identity/v3/users/4dab551c284242df98e62efdc5de8de7 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 03:59:42.286464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:42.288404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:42.305585 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 863/1701] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:42.307384 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38769c22-bdef-42ae-9472-583d50873e99 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:42.307817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38769c22-bdef-42ae-9472-583d50873e99 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:42.308225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38769c22-bdef-42ae-9472-583d50873e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:42.370296 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-38769c22-bdef-42ae-9472-583d50873e99 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:42.371646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-38769c22-bdef-42ae-9472-583d50873e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:42.461414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:42.500871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 864/1702] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:42.503476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295ae7e-d8a5-4db9-8cd4-e9c726602e7d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:42.504128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382fb11-c190-44a1-a3e3-1951c07d702d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:42.504852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382fb11-c190-44a1-a3e3-1951c07d702d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:42.505492 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382fb11-c190-44a1-a3e3-1951c07d702d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:42.509123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 865/1703] 10.222.0.46 () {54 vars in 912 bytes} [Thu Jul 28 03:59:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:59:42.510311 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382fb11-c190-44a1-a3e3-1951c07d702d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:42.512227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3382fb11-c190-44a1-a3e3-1951c07d702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:42.515768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c67295-fd6c-4646-85f1-45bc6c78dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:42.516487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c67295-fd6c-4646-85f1-45bc6c78dd4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:42.516770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c67295-fd6c-4646-85f1-45bc6c78dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:42.517045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c67295-fd6c-4646-85f1-45bc6c78dd4f admin admin] PATH_INFO: `/v3/projects/6ee6aa0e2c974038a9d6d244b3d179b7/users/ede50f1544104ce189788585b005181b/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:42.563587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c67295-fd6c-4646-85f1-45bc6c78dd4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee6aa0e2c974038a9d6d244b3d179b7, user_id=ede50f1544104ce189788585b005181b, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:42.567216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c67295-fd6c-4646-85f1-45bc6c78dd4f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:42.604660 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 839/1704] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:42 2022] PUT /identity/v3/projects/6ee6aa0e2c974038a9d6d244b3d179b7/users/ede50f1544104ce189788585b005181b/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:42.609429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab7950-85df-40db-a1ce-50001e9239f9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:42.610815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab7950-85df-40db-a1ce-50001e9239f9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:42.611473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab7950-85df-40db-a1ce-50001e9239f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:42.733642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382fb11-c190-44a1-a3e3-1951c07d702d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:42.748714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8aab7950-85df-40db-a1ce-50001e9239f9 None None] MFA Rules not processed for user `6e31408bfea4412d8f68aea9b3c7528d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:42.753411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8aab7950-85df-40db-a1ce-50001e9239f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:42.917722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047bc1f-b92a-4db7-9a54-2199e4dec54a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:42.918378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047bc1f-b92a-4db7-9a54-2199e4dec54a service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:42.918665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047bc1f-b92a-4db7-9a54-2199e4dec54a service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:42.918982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047bc1f-b92a-4db7-9a54-2199e4dec54a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:42.939822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 840/1705] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 03:59:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:42.941072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab7950-85df-40db-a1ce-50001e9239f9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:43.150870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab7950-85df-40db-a1ce-50001e9239f9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:43.155521 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047bc1f-b92a-4db7-9a54-2199e4dec54a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:43.157013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047bc1f-b92a-4db7-9a54-2199e4dec54a service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:43.159556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f6403-d454-46f7-a616-7f79e87544de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:43.162519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f6403-d454-46f7-a616-7f79e87544de admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:43.163541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f6403-d454-46f7-a616-7f79e87544de admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:43.164819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f6403-d454-46f7-a616-7f79e87544de admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:43.166920 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f6403-d454-46f7-a616-7f79e87544de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:43.173300 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f6403-d454-46f7-a616-7f79e87544de admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:43.175636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:43.201853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 841/1706] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:43.212960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f6403-d454-46f7-a616-7f79e87544de admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:43.214167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 866/1707] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:43.217160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d790211-198e-43ed-a4cd-781a61e5c207 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:43.217458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d790211-198e-43ed-a4cd-781a61e5c207 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:43.217740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d790211-198e-43ed-a4cd-781a61e5c207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:43.230571 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f6403-d454-46f7-a616-7f79e87544de admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:43.240324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23b404-dbc8-4bd5-bf95-da2980073003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:43.241303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23b404-dbc8-4bd5-bf95-da2980073003 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:43.243367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23b404-dbc8-4bd5-bf95-da2980073003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:43.243887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23b404-dbc8-4bd5-bf95-da2980073003 admin admin] PATH_INFO: `/v3/projects/6ee6aa0e2c974038a9d6d244b3d179b7/users/ede50f1544104ce189788585b005181b/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:43.254100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23b404-dbc8-4bd5-bf95-da2980073003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee6aa0e2c974038a9d6d244b3d179b7, user_id=ede50f1544104ce189788585b005181b, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:43.263679 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23b404-dbc8-4bd5-bf95-da2980073003 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:43.302435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 842/1708] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:43 2022] PUT /identity/v3/projects/6ee6aa0e2c974038a9d6d244b3d179b7/users/ede50f1544104ce189788585b005181b/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:43.312491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7d790211-198e-43ed-a4cd-781a61e5c207 None None] MFA Rules not processed for user `7112379fb6794027a7c1519b2f77a5d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:43.313362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df56d2-9ed6-4a07-a000-b424bed503c8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:43.313792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df56d2-9ed6-4a07-a000-b424bed503c8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:43.314674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7d790211-198e-43ed-a4cd-781a61e5c207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:43.315146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df56d2-9ed6-4a07-a000-b424bed503c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:43.406162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-25df56d2-9ed6-4a07-a000-b424bed503c8 None None] MFA Rules not processed for user `ede50f1544104ce189788585b005181b`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:43.412393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-25df56d2-9ed6-4a07-a000-b424bed503c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:43.459997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:43.529739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 867/1709] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:43.533531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d790211-198e-43ed-a4cd-781a61e5c207 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:43.540792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7d790211-198e-43ed-a4cd-781a61e5c207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:43.631035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 843/1710] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:43.757089 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d790211-198e-43ed-a4cd-781a61e5c207 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:43.766777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee82a55-a436-4adc-98c0-68d7e7419f12 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:43.767663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee82a55-a436-4adc-98c0-68d7e7419f12 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:43.768221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee82a55-a436-4adc-98c0-68d7e7419f12 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:43.770735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4c4a6-1bf1-4e77-b944-3629f720313f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:43.771486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4c4a6-1bf1-4e77-b944-3629f720313f service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:43.771874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4c4a6-1bf1-4e77-b944-3629f720313f service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:43.772097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4c4a6-1bf1-4e77-b944-3629f720313f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:43.772593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 844/1711] 10.222.0.46 () {54 vars in 912 bytes} [Thu Jul 28 03:59:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 03:59:43.782220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebadd1-7be8-44c0-b8b7-16a20dfe189b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:43.782518 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebadd1-7be8-44c0-b8b7-16a20dfe189b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:43.782948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebadd1-7be8-44c0-b8b7-16a20dfe189b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:43.792761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4c4a6-1bf1-4e77-b944-3629f720313f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:43.799892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4c4a6-1bf1-4e77-b944-3629f720313f service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:43.824432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 868/1712] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:43.826155 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4c4a6-1bf1-4e77-b944-3629f720313f service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:43.831790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6dd4c4a6-1bf1-4e77-b944-3629f720313f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:43.848588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e5ebadd1-7be8-44c0-b8b7-16a20dfe189b None None] MFA Rules not processed for user `6e31408bfea4412d8f68aea9b3c7528d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:43.853742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e5ebadd1-7be8-44c0-b8b7-16a20dfe189b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:43.879497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 845/1713] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 03:59:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:43.906757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebadd1-7be8-44c0-b8b7-16a20dfe189b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:43.922482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebadd1-7be8-44c0-b8b7-16a20dfe189b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:43.936097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe69146-f659-468c-b7c7-7c91348cb000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:43.938013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe69146-f659-468c-b7c7-7c91348cb000 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:43.938305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe69146-f659-468c-b7c7-7c91348cb000 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:43.938591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe69146-f659-468c-b7c7-7c91348cb000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:43.983995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe69146-f659-468c-b7c7-7c91348cb000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:44.005104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe69146-f659-468c-b7c7-7c91348cb000 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:44.013881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4c4a6-1bf1-4e77-b944-3629f720313f service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:44.023014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae5e86-ce4b-4e93-8628-c461e7a8e22c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:44.023958 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae5e86-ce4b-4e93-8628-c461e7a8e22c service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:44.024317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae5e86-ce4b-4e93-8628-c461e7a8e22c service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:44.024865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae5e86-ce4b-4e93-8628-c461e7a8e22c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:44.027056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 846/1714] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:44.043685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae5e86-ce4b-4e93-8628-c461e7a8e22c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:44.045481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae5e86-ce4b-4e93-8628-c461e7a8e22c service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:44.068554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 869/1715] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:44.280528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe69146-f659-468c-b7c7-7c91348cb000 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:44.303987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe69146-f659-468c-b7c7-7c91348cb000 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:44.326114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705fd9e-fdec-4880-b0b8-e1601dd09a9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:44.327305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705fd9e-fdec-4880-b0b8-e1601dd09a9b service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:44.327606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705fd9e-fdec-4880-b0b8-e1601dd09a9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:44.327891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705fd9e-fdec-4880-b0b8-e1601dd09a9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:44.351784 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5705fd9e-fdec-4880-b0b8-e1601dd09a9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:44.500287 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae5e86-ce4b-4e93-8628-c461e7a8e22c service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:44.502343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4fae5e86-ce4b-4e93-8628-c461e7a8e22c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:44.575885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705fd9e-fdec-4880-b0b8-e1601dd09a9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:44.577803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705fd9e-fdec-4880-b0b8-e1601dd09a9b service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:44.612425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 847/1716] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 03:59:44.691443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae5e86-ce4b-4e93-8628-c461e7a8e22c service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:44.701043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:44.701721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:44.702050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:44.702334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] PATH_INFO: `/v3/projects/726a1ad14d374061a5db26470f86fdf7` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:44.707427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=726a1ad14d374061a5db26470f86fdf7)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:44.725019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:44.823383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:44.835338 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '726a1ad14d374061a5db26470f86fdf7', 'request_id': 'req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:44.850771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '726a1ad14d374061a5db26470f86fdf7', 'request_id': 'req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:44.859671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '726a1ad14d374061a5db26470f86fdf7', 'request_id': 'req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:44.877931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] The token cache is being invalidate because project 726a1ad14d374061a5db26470f86fdf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 03:59:44.878647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:44.879439 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c7397f0c-9a52-446f-959a-ad0a5a8a7cf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:44.881534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 870/1717] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 03:59:44 2022] DELETE /identity/v3/projects/726a1ad14d374061a5db26470f86fdf7 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 03:59:44.888372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fbfc5-2bbe-4b35-8ec8-58b67b7396f7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:44.888372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fbfc5-2bbe-4b35-8ec8-58b67b7396f7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:44.888372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fbfc5-2bbe-4b35-8ec8-58b67b7396f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:44.936563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ab8fbfc5-2bbe-4b35-8ec8-58b67b7396f7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:44.937868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ab8fbfc5-2bbe-4b35-8ec8-58b67b7396f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:45.015818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:45.077291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661176bc-36a8-4e98-9ba2-a0c8d3a56db8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:45.077550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661176bc-36a8-4e98-9ba2-a0c8d3a56db8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:45.077844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661176bc-36a8-4e98-9ba2-a0c8d3a56db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:45.081458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 848/1718] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:45.113598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-661176bc-36a8-4e98-9ba2-a0c8d3a56db8 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:45.114874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-661176bc-36a8-4e98-9ba2-a0c8d3a56db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:45.141940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fbfc5-2bbe-4b35-8ec8-58b67b7396f7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:45.150066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 871/1719] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:45.170237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fbfc5-2bbe-4b35-8ec8-58b67b7396f7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:45.202749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bb4ad-1668-44de-b4ed-4241ad823bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:45.215687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bb4ad-1668-44de-b4ed-4241ad823bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:45.215687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bb4ad-1668-44de-b4ed-4241ad823bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:45.215687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bb4ad-1668-44de-b4ed-4241ad823bfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:45.215687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bb4ad-1668-44de-b4ed-4241ad823bfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:45.215687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bb4ad-1668-44de-b4ed-4241ad823bfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:45.215687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bb4ad-1668-44de-b4ed-4241ad823bfb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:45.216880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f391d1-666a-4e15-8414-c475101dc143 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:45.217167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f391d1-666a-4e15-8414-c475101dc143 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:45.217446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f391d1-666a-4e15-8414-c475101dc143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:45.303465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 849/1720] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:45.305745 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bb4ad-1668-44de-b4ed-4241ad823bfb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:45.313132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-289bb4ad-1668-44de-b4ed-4241ad823bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:45.321841 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f0f391d1-666a-4e15-8414-c475101dc143 None None] MFA Rules not processed for user `72dbbd19e4454bee84fe5bc797497bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:45.323334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f0f391d1-666a-4e15-8414-c475101dc143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:45.367806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bb4ad-1668-44de-b4ed-4241ad823bfb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:45.378019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:45.378693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:45.379003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:45.379282 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] PATH_INFO: `/v3/users/f7caf8adb43340e0a5521863113a3daa` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:45.383281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7caf8adb43340e0a5521863113a3daa)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:45.385277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:45.466401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 872/1721] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:45.467543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f391d1-666a-4e15-8414-c475101dc143 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:45.469047 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f0f391d1-666a-4e15-8414-c475101dc143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:45.542885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7caf8adb43340e0a5521863113a3daa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:45.545561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7caf8adb43340e0a5521863113a3daa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:45.564723 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7caf8adb43340e0a5521863113a3daa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:45.596426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7caf8adb43340e0a5521863113a3daa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:45.639232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7caf8adb43340e0a5521863113a3daa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:45.668915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 850/1722] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 03:59:45 2022] DELETE /identity/v3/users/f7caf8adb43340e0a5521863113a3daa => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:45.679490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f391d1-666a-4e15-8414-c475101dc143 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:45.680027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:45.720972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:45.806139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:45.854677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03011b87-eb9a-4085-9816-c5fe848ba740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:45.855353 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03011b87-eb9a-4085-9816-c5fe848ba740 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:45.855642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03011b87-eb9a-4085-9816-c5fe848ba740 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:45.855923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03011b87-eb9a-4085-9816-c5fe848ba740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:45.863256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-03011b87-eb9a-4085-9816-c5fe848ba740 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:45.938528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daad8a6-e4a6-47b8-b3c1-5e3ab165fb5f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:45.960232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298603f-da8a-42f2-903f-f11474797fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:45.961007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298603f-da8a-42f2-903f-f11474797fde admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:45.964223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298603f-da8a-42f2-903f-f11474797fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:45.964630 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298603f-da8a-42f2-903f-f11474797fde admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:45.965730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298603f-da8a-42f2-903f-f11474797fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:45.977223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298603f-da8a-42f2-903f-f11474797fde admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:46.013352 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03011b87-eb9a-4085-9816-c5fe848ba740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:46.019167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03011b87-eb9a-4085-9816-c5fe848ba740 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:46.042747 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 873/1723] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:46.046388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69cc2d-597c-47ca-a2ce-7a29b271e258 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:46.046689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69cc2d-597c-47ca-a2ce-7a29b271e258 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:46.049759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69cc2d-597c-47ca-a2ce-7a29b271e258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:46.060893 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 851/1724] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:45 2022] POST /identity/v3/projects => generated 371 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:46.062454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3982f0-f7f8-4979-931c-3eed12c2c692 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:46.062754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3982f0-f7f8-4979-931c-3eed12c2c692 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:46.064207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3982f0-f7f8-4979-931c-3eed12c2c692 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:46.065693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 852/1725] 10.222.0.46 () {54 vars in 961 bytes} [Thu Jul 28 03:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:59:46.066648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f210ea2-fa1d-4b5c-83c7-83b0de1f9d82 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:46.066967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f210ea2-fa1d-4b5c-83c7-83b0de1f9d82 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:46.067250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f210ea2-fa1d-4b5c-83c7-83b0de1f9d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:46.107761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1b69cc2d-597c-47ca-a2ce-7a29b271e258 None None] MFA Rules not processed for user `72dbbd19e4454bee84fe5bc797497bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:46.109069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1b69cc2d-597c-47ca-a2ce-7a29b271e258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:46.121934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1f210ea2-fa1d-4b5c-83c7-83b0de1f9d82 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:46.123232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1f210ea2-fa1d-4b5c-83c7-83b0de1f9d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:46.191042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:46.234746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 874/1726] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:46.238613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69cc2d-597c-47ca-a2ce-7a29b271e258 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:46.242881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1b69cc2d-597c-47ca-a2ce-7a29b271e258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:46.296646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 853/1727] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:46.297698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f210ea2-fa1d-4b5c-83c7-83b0de1f9d82 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:46.322443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f210ea2-fa1d-4b5c-83c7-83b0de1f9d82 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:46.336376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb864d8b-e54f-42d0-bf0b-28036b74fd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:46.337014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb864d8b-e54f-42d0-bf0b-28036b74fd40 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:46.337298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb864d8b-e54f-42d0-bf0b-28036b74fd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:46.337571 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb864d8b-e54f-42d0-bf0b-28036b74fd40 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:46.338247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb864d8b-e54f-42d0-bf0b-28036b74fd40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:46.340223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb864d8b-e54f-42d0-bf0b-28036b74fd40 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:46.392473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 854/1728] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:46 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:46.393832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472852e4-c331-4016-8d57-a9d1e60bcccf None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:46.394121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472852e4-c331-4016-8d57-a9d1e60bcccf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:46.394485 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472852e4-c331-4016-8d57-a9d1e60bcccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:46.416840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69cc2d-597c-47ca-a2ce-7a29b271e258 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:46.433655 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c81ee-322b-4489-8539-be37c1b439df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:46.437615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c81ee-322b-4489-8539-be37c1b439df service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:46.437615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c81ee-322b-4489-8539-be37c1b439df service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:46.440682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c81ee-322b-4489-8539-be37c1b439df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:46.445478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a48c81ee-322b-4489-8539-be37c1b439df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:46.451940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-472852e4-c331-4016-8d57-a9d1e60bcccf None None] MFA Rules not processed for user `551281a31ce24e2f9b81b36f69419311`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:46.456557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-472852e4-c331-4016-8d57-a9d1e60bcccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:46.479509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c81ee-322b-4489-8539-be37c1b439df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:46.481026 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 855/1729] 10.222.0.46 () {58 vars in 1058 bytes} [Thu Jul 28 03:59:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:46.482447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c81ee-322b-4489-8539-be37c1b439df service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:46.482609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472852e4-c331-4016-8d57-a9d1e60bcccf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:46.487441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-472852e4-c331-4016-8d57-a9d1e60bcccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:46.541546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 875/1730] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:46.542558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c81ee-322b-4489-8539-be37c1b439df service placement] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:46.552132 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a48c81ee-322b-4489-8539-be37c1b439df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:46.626631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:46.728248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c81ee-322b-4489-8539-be37c1b439df service placement] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:46.743331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f094d37-d5f8-4f9a-ac08-48032002ed59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:46.743984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f094d37-d5f8-4f9a-ac08-48032002ed59 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:46.744270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f094d37-d5f8-4f9a-ac08-48032002ed59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:46.744552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f094d37-d5f8-4f9a-ac08-48032002ed59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:46.799255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f094d37-d5f8-4f9a-ac08-48032002ed59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:46.800805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f094d37-d5f8-4f9a-ac08-48032002ed59 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:46.831551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472852e4-c331-4016-8d57-a9d1e60bcccf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:46.836668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 876/1731] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:46.837661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f094d37-d5f8-4f9a-ac08-48032002ed59 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:46.857985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f285-0cea-496c-80bc-2bc5ff6a2235 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:46.858694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f285-0cea-496c-80bc-2bc5ff6a2235 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:46.858931 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f285-0cea-496c-80bc-2bc5ff6a2235 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:46.859210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f285-0cea-496c-80bc-2bc5ff6a2235 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:46.869691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-acd9f285-0cea-496c-80bc-2bc5ff6a2235 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:46.875472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f094d37-d5f8-4f9a-ac08-48032002ed59 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:46.888722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616fbf-3a55-45b5-bcb0-ebd8a43823a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:46.889368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616fbf-3a55-45b5-bcb0-ebd8a43823a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:46.889652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616fbf-3a55-45b5-bcb0-ebd8a43823a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:46.889930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616fbf-3a55-45b5-bcb0-ebd8a43823a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:46.890633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0616fbf-3a55-45b5-bcb0-ebd8a43823a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:46.892828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0616fbf-3a55-45b5-bcb0-ebd8a43823a5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:46.895084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9f285-0cea-496c-80bc-2bc5ff6a2235 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:46.896606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9f285-0cea-496c-80bc-2bc5ff6a2235 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:46.917079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 877/1732] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:46.917947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616fbf-3a55-45b5-bcb0-ebd8a43823a5 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:46.931192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 856/1733] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:46.932196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f285-0cea-496c-80bc-2bc5ff6a2235 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:46.969654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616fbf-3a55-45b5-bcb0-ebd8a43823a5 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:46.971377 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f285-0cea-496c-80bc-2bc5ff6a2235 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:46.984076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8d231-4f41-427c-9d02-33b90ab242ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:46.985639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8d231-4f41-427c-9d02-33b90ab242ba admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:46.986219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8d231-4f41-427c-9d02-33b90ab242ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:46.986452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8d231-4f41-427c-9d02-33b90ab242ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:46.988067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf3cf7-08ce-4ccc-b8ff-ba8e74cf744a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:46.988998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8d231-4f41-427c-9d02-33b90ab242ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:46.989794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3cf7-08ce-4ccc-b8ff-ba8e74cf744a service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:46.990224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3cf7-08ce-4ccc-b8ff-ba8e74cf744a service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:46.992061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8d231-4f41-427c-9d02-33b90ab242ba admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:46.992529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3cf7-08ce-4ccc-b8ff-ba8e74cf744a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:47.013388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf3cf7-08ce-4ccc-b8ff-ba8e74cf744a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:47.015101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf3cf7-08ce-4ccc-b8ff-ba8e74cf744a service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:47.021908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 857/1734] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:47.025712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8d231-4f41-427c-9d02-33b90ab242ba admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:47.047865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8d231-4f41-427c-9d02-33b90ab242ba admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:47.049712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 878/1735] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:47.065980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c98ddb-5761-4124-b29d-7f9f12d8f9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:47.081919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98ddb-5761-4124-b29d-7f9f12d8f9fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:47.082349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98ddb-5761-4124-b29d-7f9f12d8f9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:47.082643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98ddb-5761-4124-b29d-7f9f12d8f9fb admin admin] PATH_INFO: `/v3/projects/c4082865fb164d39ba7257b5fd308ab5/users/0bf42d8e1df64661af5fb0f286df8881/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:47.106723 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c98ddb-5761-4124-b29d-7f9f12d8f9fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4082865fb164d39ba7257b5fd308ab5, user_id=0bf42d8e1df64661af5fb0f286df8881, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:47.118381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c98ddb-5761-4124-b29d-7f9f12d8f9fb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:47.152217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 858/1736] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:47 2022] PUT /identity/v3/projects/c4082865fb164d39ba7257b5fd308ab5/users/0bf42d8e1df64661af5fb0f286df8881/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:47.170852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5c05d-8d95-4207-a136-16ae8ceba109 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:47.170852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5c05d-8d95-4207-a136-16ae8ceba109 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:47.170852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5c05d-8d95-4207-a136-16ae8ceba109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:47.257952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e8d5c05d-8d95-4207-a136-16ae8ceba109 None None] MFA Rules not processed for user `0bf42d8e1df64661af5fb0f286df8881`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:47.262846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e8d5c05d-8d95-4207-a136-16ae8ceba109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:47.505773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 879/1737] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:47.517583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c98ddb-5761-4124-b29d-7f9f12d8f9fb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:47.519989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a3c98ddb-5761-4124-b29d-7f9f12d8f9fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:47.679748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:47.718732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c98ddb-5761-4124-b29d-7f9f12d8f9fb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:47.728097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5908940-5b49-4ec2-bccf-38219a579803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:47.730607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5908940-5b49-4ec2-bccf-38219a579803 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:47.731320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5908940-5b49-4ec2-bccf-38219a579803 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:47.731815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5908940-5b49-4ec2-bccf-38219a579803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:47.765190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5908940-5b49-4ec2-bccf-38219a579803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:47.768599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5908940-5b49-4ec2-bccf-38219a579803 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:47.808793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 859/1738] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:49.181867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5c05d-8d95-4207-a136-16ae8ceba109 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:49.183521 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e8d5c05d-8d95-4207-a136-16ae8ceba109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:49.451560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5c05d-8d95-4207-a136-16ae8ceba109 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:49.462076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:49.464191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:49.464643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:49.466727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] PATH_INFO: `/v3/projects/017b0d1b30a44f5887b0a23e7db3a489` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:49.470493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] RBAC: Authorizing `identity:delete_project(project_id=017b0d1b30a44f5887b0a23e7db3a489)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:49.476403 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:49.630492 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:49.691436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017b0d1b30a44f5887b0a23e7db3a489', 'request_id': 'req-4be4d66e-b471-4665-9dc0-53a01ecc140b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:49.721777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017b0d1b30a44f5887b0a23e7db3a489', 'request_id': 'req-4be4d66e-b471-4665-9dc0-53a01ecc140b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:49.741735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017b0d1b30a44f5887b0a23e7db3a489', 'request_id': 'req-4be4d66e-b471-4665-9dc0-53a01ecc140b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:49.772099 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] The token cache is being invalidate because project 017b0d1b30a44f5887b0a23e7db3a489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 03:59:49.773138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:49.773138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4be4d66e-b471-4665-9dc0-53a01ecc140b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:49.777649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 880/1739] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 03:59:49 2022] DELETE /identity/v3/projects/017b0d1b30a44f5887b0a23e7db3a489 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:49.781974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5908940-5b49-4ec2-bccf-38219a579803 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:49.814553 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5908940-5b49-4ec2-bccf-38219a579803 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:49.976872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e9e97-da74-47c5-b22f-c20a7b82f6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:49.980468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e9e97-da74-47c5-b22f-c20a7b82f6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:49.991471 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e9e97-da74-47c5-b22f-c20a7b82f6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:49.991471 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e9e97-da74-47c5-b22f-c20a7b82f6ba admin admin] PATH_INFO: `/v3/users/ede50f1544104ce189788585b005181b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:49.998973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e9e97-da74-47c5-b22f-c20a7b82f6ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede50f1544104ce189788585b005181b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:50.010921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd55a7-a70b-42c9-ac06-408f9bd29a00 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:50.011376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd55a7-a70b-42c9-ac06-408f9bd29a00 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:50.011806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd55a7-a70b-42c9-ac06-408f9bd29a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:50.013065 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e9e97-da74-47c5-b22f-c20a7b82f6ba admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:50.082383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-34dd55a7-a70b-42c9-ac06-408f9bd29a00 None None] MFA Rules not processed for user `72dbbd19e4454bee84fe5bc797497bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:50.089198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-34dd55a7-a70b-42c9-ac06-408f9bd29a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:50.199881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0c7e9e97-da74-47c5-b22f-c20a7b82f6ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede50f1544104ce189788585b005181b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:50.200758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0c7e9e97-da74-47c5-b22f-c20a7b82f6ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede50f1544104ce189788585b005181b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:50.219574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0c7e9e97-da74-47c5-b22f-c20a7b82f6ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede50f1544104ce189788585b005181b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:50.259284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0c7e9e97-da74-47c5-b22f-c20a7b82f6ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede50f1544104ce189788585b005181b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:50.286081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0c7e9e97-da74-47c5-b22f-c20a7b82f6ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede50f1544104ce189788585b005181b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:50.305862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 860/1740] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 03:59:49 2022] DELETE /identity/v3/users/ede50f1544104ce189788585b005181b => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:50.326091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adac59-70d2-49d9-8b15-9f583c41e3b5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:50.326394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adac59-70d2-49d9-8b15-9f583c41e3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:50.326986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adac59-70d2-49d9-8b15-9f583c41e3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:50.337730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 881/1741] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:50.405116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c3adac59-70d2-49d9-8b15-9f583c41e3b5 None None] MFA Rules not processed for user `7112379fb6794027a7c1519b2f77a5d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:50.411823 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c3adac59-70d2-49d9-8b15-9f583c41e3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:50.642413 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 861/1742] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:51.135448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332c42-7ee5-4486-b7ce-b5665a5f490d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:51.135746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332c42-7ee5-4486-b7ce-b5665a5f490d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:51.136036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332c42-7ee5-4486-b7ce-b5665a5f490d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:51.266431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4b332c42-7ee5-4486-b7ce-b5665a5f490d None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:51.269026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4b332c42-7ee5-4486-b7ce-b5665a5f490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:51.427291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 882/1743] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:51.591423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adac59-70d2-49d9-8b15-9f583c41e3b5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:51.593647 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c3adac59-70d2-49d9-8b15-9f583c41e3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:51.608175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:51.770747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adac59-70d2-49d9-8b15-9f583c41e3b5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:51.776021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240cb7e-9e5c-4a18-aa21-ad45e6ff1ea5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:51.776644 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240cb7e-9e5c-4a18-aa21-ad45e6ff1ea5 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:51.776926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240cb7e-9e5c-4a18-aa21-ad45e6ff1ea5 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:51.777725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240cb7e-9e5c-4a18-aa21-ad45e6ff1ea5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:51.791991 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240cb7e-9e5c-4a18-aa21-ad45e6ff1ea5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:51.795236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240cb7e-9e5c-4a18-aa21-ad45e6ff1ea5 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:51.820031 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 862/1744] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 03:59:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 03:59:51.932444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332c42-7ee5-4486-b7ce-b5665a5f490d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:51.934820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4b332c42-7ee5-4486-b7ce-b5665a5f490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:52.103539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:52.239075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332c42-7ee5-4486-b7ce-b5665a5f490d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:52.256277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:52.258207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:52.258639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:52.260269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] PATH_INFO: `/v3/users/0bf42d8e1df64661af5fb0f286df8881` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:52.263091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf42d8e1df64661af5fb0f286df8881)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:52.265185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:52.397339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf42d8e1df64661af5fb0f286df8881'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:52.398131 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf42d8e1df64661af5fb0f286df8881'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:52.446260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf42d8e1df64661af5fb0f286df8881'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:52.481815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf42d8e1df64661af5fb0f286df8881'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:52.539759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf42d8e1df64661af5fb0f286df8881'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:52.551741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 883/1745] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 03:59:51 2022] DELETE /identity/v3/users/0bf42d8e1df64661af5fb0f286df8881 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:52.556808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c83ee2-8adc-4149-b4fa-750a1f5615dc None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:52.557346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c83ee2-8adc-4149-b4fa-750a1f5615dc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:52.557816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c83ee2-8adc-4149-b4fa-750a1f5615dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:52.618836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-07c83ee2-8adc-4149-b4fa-750a1f5615dc None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:52.620928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-07c83ee2-8adc-4149-b4fa-750a1f5615dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:52.800590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 863/1746] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:52.940038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:52.942092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:53.102947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055af3d-144a-4652-bd21-52e253528674 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:53.114090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5d4bc-e758-44e3-9d50-79f13db013db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:53.116454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5d4bc-e758-44e3-9d50-79f13db013db service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:53.116919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5d4bc-e758-44e3-9d50-79f13db013db service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:53.117366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5d4bc-e758-44e3-9d50-79f13db013db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:53.146737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5d4bc-e758-44e3-9d50-79f13db013db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:53.148545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5d4bc-e758-44e3-9d50-79f13db013db service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:53.174701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 884/1747] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:55.486992 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940af99d-255e-4785-b814-20a961aa59e4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:55.487524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940af99d-255e-4785-b814-20a961aa59e4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:55.487699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940af99d-255e-4785-b814-20a961aa59e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:55.550391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-940af99d-255e-4785-b814-20a961aa59e4 None None] MFA Rules not processed for user `6e31408bfea4412d8f68aea9b3c7528d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:55.553250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-940af99d-255e-4785-b814-20a961aa59e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:55.615976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 864/1748] 10.222.0.39 () {58 vars in 1015 bytes} [Thu Jul 28 03:59:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:55.885828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5d4bc-e758-44e3-9d50-79f13db013db service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:55.887532 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1ab5d4bc-e758-44e3-9d50-79f13db013db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:55.917954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5d4bc-e758-44e3-9d50-79f13db013db service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:55.925868 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:55.926826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:55.927263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:55.927689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] PATH_INFO: `/v3/projects/6ee6aa0e2c974038a9d6d244b3d179b7` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:55.931708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee6aa0e2c974038a9d6d244b3d179b7)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:55.933971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:56.153123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee6aa0e2c974038a9d6d244b3d179b7', 'request_id': 'req-375fb806-8ef9-4c84-a054-e580b7961cbd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:56.155304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:56.174681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee6aa0e2c974038a9d6d244b3d179b7', 'request_id': 'req-375fb806-8ef9-4c84-a054-e580b7961cbd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:56.213239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee6aa0e2c974038a9d6d244b3d179b7', 'request_id': 'req-375fb806-8ef9-4c84-a054-e580b7961cbd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:56.268906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] The token cache is being invalidate because project 6ee6aa0e2c974038a9d6d244b3d179b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 03:59:56.268906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:56.268906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:56.273458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 885/1749] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 03:59:55 2022] DELETE /identity/v3/projects/6ee6aa0e2c974038a9d6d244b3d179b7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:56.277364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe7ef-05ce-4f27-b13b-5c5ab1933a75 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:56.277893 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe7ef-05ce-4f27-b13b-5c5ab1933a75 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:56.278181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe7ef-05ce-4f27-b13b-5c5ab1933a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:56.312687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:56.315036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:56.415583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7e1fe7ef-05ce-4f27-b13b-5c5ab1933a75 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:56.427363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7e1fe7ef-05ce-4f27-b13b-5c5ab1933a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:56.682033 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fb806-8ef9-4c84-a054-e580b7961cbd admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:56.694050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7e06e-e16f-4b9a-85a7-03daac5b2b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:56.694960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7e06e-e16f-4b9a-85a7-03daac5b2b9a service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:56.696255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7e06e-e16f-4b9a-85a7-03daac5b2b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:56.697570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7e06e-e16f-4b9a-85a7-03daac5b2b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:56.703264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9fd7e06e-e16f-4b9a-85a7-03daac5b2b9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:56.728616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7e06e-e16f-4b9a-85a7-03daac5b2b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:56.734681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7e06e-e16f-4b9a-85a7-03daac5b2b9a service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:56.761192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 886/1750] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 03:59:56.796291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7e06e-e16f-4b9a-85a7-03daac5b2b9a service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:56.798355 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9fd7e06e-e16f-4b9a-85a7-03daac5b2b9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:56.827036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 865/1751] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 544 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:56.828913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d463d-3186-421f-8a91-27c51af43b24 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:56.829856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d463d-3186-421f-8a91-27c51af43b24 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:56.830374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d463d-3186-421f-8a91-27c51af43b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:56.837052 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7e06e-e16f-4b9a-85a7-03daac5b2b9a service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:56.853959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5505b80-00f7-46e4-815d-5a5881d64bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:56.856944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5505b80-00f7-46e4-815d-5a5881d64bfe service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:56.857240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5505b80-00f7-46e4-815d-5a5881d64bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:56.857526 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5505b80-00f7-46e4-815d-5a5881d64bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:56.900856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5505b80-00f7-46e4-815d-5a5881d64bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:56.909643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5505b80-00f7-46e4-815d-5a5881d64bfe service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:56.917911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-863d463d-3186-421f-8a91-27c51af43b24 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:56.921542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-863d463d-3186-421f-8a91-27c51af43b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:56.957496 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 887/1752] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 03:59:56.971846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 866/1753] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:56.976616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5505b80-00f7-46e4-815d-5a5881d64bfe service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:57.015541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5505b80-00f7-46e4-815d-5a5881d64bfe service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:57.036691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd4e67-420c-4ac3-bec2-a2f1057f4830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:57.039908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd4e67-420c-4ac3-bec2-a2f1057f4830 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:57.040405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd4e67-420c-4ac3-bec2-a2f1057f4830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:57.041416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd4e67-420c-4ac3-bec2-a2f1057f4830 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:57.043413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd4e67-420c-4ac3-bec2-a2f1057f4830 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:57.044927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd4e67-420c-4ac3-bec2-a2f1057f4830 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:57.046745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd4e67-420c-4ac3-bec2-a2f1057f4830 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:57.092740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 888/1754] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:57.100217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d463d-3186-421f-8a91-27c51af43b24 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:57.107386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:57.124964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d463d-3186-421f-8a91-27c51af43b24 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:57.137762 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd4e67-420c-4ac3-bec2-a2f1057f4830 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:57.144834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-adbd4e67-420c-4ac3-bec2-a2f1057f4830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:57.153670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d73a06-09bb-422f-aca6-cf9c9e5eeca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:57.159275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d73a06-09bb-422f-aca6-cf9c9e5eeca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:57.163806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d73a06-09bb-422f-aca6-cf9c9e5eeca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:57.165050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d73a06-09bb-422f-aca6-cf9c9e5eeca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:57.168196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d73a06-09bb-422f-aca6-cf9c9e5eeca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:57.181339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d73a06-09bb-422f-aca6-cf9c9e5eeca1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:57.205000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd4e67-420c-4ac3-bec2-a2f1057f4830 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:57.219708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:57.222612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:57.226137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:57.226486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] PATH_INFO: `/v3/projects/c4082865fb164d39ba7257b5fd308ab5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:57.228973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4082865fb164d39ba7257b5fd308ab5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:57.240070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:57.316459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 867/1755] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:57 2022] POST /identity/v3/projects => generated 369 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 03:59:57.319479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d73a06-09bb-422f-aca6-cf9c9e5eeca1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:57.427835 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4082865fb164d39ba7257b5fd308ab5', 'request_id': 'req-bc26d8f0-9304-4569-91a7-1900c3d5ea63'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:57.443628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4082865fb164d39ba7257b5fd308ab5', 'request_id': 'req-bc26d8f0-9304-4569-91a7-1900c3d5ea63'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:57.457703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4082865fb164d39ba7257b5fd308ab5', 'request_id': 'req-bc26d8f0-9304-4569-91a7-1900c3d5ea63'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:57.513730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] The token cache is being invalidate because project c4082865fb164d39ba7257b5fd308ab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 03:59:57.514044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:57.517472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bc26d8f0-9304-4569-91a7-1900c3d5ea63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 03:59:57.528082 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 889/1756] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 03:59:57 2022] DELETE /identity/v3/projects/c4082865fb164d39ba7257b5fd308ab5 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:57.529724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78d525-00da-445e-bdd4-dc364ab3455d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:57.530040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78d525-00da-445e-bdd4-dc364ab3455d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:57.531045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78d525-00da-445e-bdd4-dc364ab3455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:57.662208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1e78d525-00da-445e-bdd4-dc364ab3455d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:57.664924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1e78d525-00da-445e-bdd4-dc364ab3455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:57.706841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d73a06-09bb-422f-aca6-cf9c9e5eeca1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:57.738915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:57.856741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 890/1757] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:57.864485 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba869ee-b1da-42d2-b6db-12440c32b7a1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:57.864919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba869ee-b1da-42d2-b6db-12440c32b7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:57.865304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba869ee-b1da-42d2-b6db-12440c32b7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:57.878665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a82375-aa68-4285-9857-109aa66b77f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:57.879922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a82375-aa68-4285-9857-109aa66b77f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:57.881418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a82375-aa68-4285-9857-109aa66b77f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:57.881745 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a82375-aa68-4285-9857-109aa66b77f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:57.884593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a82375-aa68-4285-9857-109aa66b77f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:57.886585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a82375-aa68-4285-9857-109aa66b77f8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:57.914852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9ba869ee-b1da-42d2-b6db-12440c32b7a1 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:57.916350 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9ba869ee-b1da-42d2-b6db-12440c32b7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:57.938912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 891/1758] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:57.943981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba869ee-b1da-42d2-b6db-12440c32b7a1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:57.964009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba869ee-b1da-42d2-b6db-12440c32b7a1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:57.974054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 868/1759] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:57 2022] POST /identity/v3/users => generated 285 bytes in 657 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:57.978771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a82375-aa68-4285-9857-109aa66b77f8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:57.985407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2dd91-8b65-4cd9-baa0-138fdf3ebf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:57.987627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-57a82375-aa68-4285-9857-109aa66b77f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:57.988911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2dd91-8b65-4cd9-baa0-138fdf3ebf46 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:57.989220 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2dd91-8b65-4cd9-baa0-138fdf3ebf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:57.989508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2dd91-8b65-4cd9-baa0-138fdf3ebf46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:58.004489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2dd91-8b65-4cd9-baa0-138fdf3ebf46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 03:59:58.004888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2dd91-8b65-4cd9-baa0-138fdf3ebf46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:58.010770 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2dd91-8b65-4cd9-baa0-138fdf3ebf46 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:58.026097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a82375-aa68-4285-9857-109aa66b77f8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:58.050934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952035e2-7bbe-4ef4-b04d-ab5c321e4939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:58.056998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952035e2-7bbe-4ef4-b04d-ab5c321e4939 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:58.057354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952035e2-7bbe-4ef4-b04d-ab5c321e4939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:58.057646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952035e2-7bbe-4ef4-b04d-ab5c321e4939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:58.058409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952035e2-7bbe-4ef4-b04d-ab5c321e4939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:58.069594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 892/1760] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 03:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 03:59:58.070601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952035e2-7bbe-4ef4-b04d-ab5c321e4939 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:58.078014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2dd91-8b65-4cd9-baa0-138fdf3ebf46 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:58.126972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 869/1761] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:58.127171 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2dd91-8b65-4cd9-baa0-138fdf3ebf46 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:58.130541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952035e2-7bbe-4ef4-b04d-ab5c321e4939 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:58.143795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76854600-26a4-46d8-8c99-dfc3a52f3627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:58.144435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76854600-26a4-46d8-8c99-dfc3a52f3627 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:58.144718 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76854600-26a4-46d8-8c99-dfc3a52f3627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:58.144998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76854600-26a4-46d8-8c99-dfc3a52f3627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:58.145679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76854600-26a4-46d8-8c99-dfc3a52f3627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:58.148532 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76854600-26a4-46d8-8c99-dfc3a52f3627 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:58.167883 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952035e2-7bbe-4ef4-b04d-ab5c321e4939 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:58.181005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a8a58-abdd-40d7-8a1f-56f4766d19e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:58.181683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a8a58-abdd-40d7-8a1f-56f4766d19e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:58.181972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a8a58-abdd-40d7-8a1f-56f4766d19e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:58.182252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a8a58-abdd-40d7-8a1f-56f4766d19e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:58.182993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a8a58-abdd-40d7-8a1f-56f4766d19e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:58.184634 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a8a58-abdd-40d7-8a1f-56f4766d19e6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:58.231672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 893/1762] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 03:59:58 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:58.235113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 870/1763] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:58.236019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76854600-26a4-46d8-8c99-dfc3a52f3627 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:58.240022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a8a58-abdd-40d7-8a1f-56f4766d19e6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:58.261000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a8a58-abdd-40d7-8a1f-56f4766d19e6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:58.353351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:58.367304 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553bf4e-a40e-4e6b-bc05-8aedb0c119c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:58.368287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553bf4e-a40e-4e6b-bc05-8aedb0c119c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:58.368717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553bf4e-a40e-4e6b-bc05-8aedb0c119c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:58.369218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553bf4e-a40e-4e6b-bc05-8aedb0c119c5 admin admin] PATH_INFO: `/v3/projects/6edd43bd2d9c4f3280cff25aa19d4fc5/users/a86c1febcbbe49a0881dfb9334cedba2/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:58.395117 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553bf4e-a40e-4e6b-bc05-8aedb0c119c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edd43bd2d9c4f3280cff25aa19d4fc5, user_id=a86c1febcbbe49a0881dfb9334cedba2, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:58.397829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553bf4e-a40e-4e6b-bc05-8aedb0c119c5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:58.428120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76854600-26a4-46d8-8c99-dfc3a52f3627 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:58.430327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 871/1764] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:58 2022] PUT /identity/v3/projects/6edd43bd2d9c4f3280cff25aa19d4fc5/users/a86c1febcbbe49a0881dfb9334cedba2/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:58.436156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c3da2-41bd-435b-a53f-c9eebc79cda7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:58.436780 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c3da2-41bd-435b-a53f-c9eebc79cda7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:58.437402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c3da2-41bd-435b-a53f-c9eebc79cda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:58.466216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-517c3da2-41bd-435b-a53f-c9eebc79cda7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:58.467067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-517c3da2-41bd-435b-a53f-c9eebc79cda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:58.639625 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af6d97-dbf3-4592-8b7b-8b34bd078123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:58.641209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af6d97-dbf3-4592-8b7b-8b34bd078123 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:58.645481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af6d97-dbf3-4592-8b7b-8b34bd078123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:58.645850 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af6d97-dbf3-4592-8b7b-8b34bd078123 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:58.648362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af6d97-dbf3-4592-8b7b-8b34bd078123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:58.654039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af6d97-dbf3-4592-8b7b-8b34bd078123 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:58.667321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 872/1765] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:58.731392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 894/1766] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 03:59:58 2022] POST /identity/v3/users => generated 288 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 03:59:58.733207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c3da2-41bd-435b-a53f-c9eebc79cda7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:58.747795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af6d97-dbf3-4592-8b7b-8b34bd078123 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:58.758914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-32af6d97-dbf3-4592-8b7b-8b34bd078123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:58.762680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c3da2-41bd-435b-a53f-c9eebc79cda7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:58.792457 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195d84d-a766-45cf-a627-7cfbf36b0751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:58.795560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d84d-a766-45cf-a627-7cfbf36b0751 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:58.805187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d84d-a766-45cf-a627-7cfbf36b0751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:58.808467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d84d-a766-45cf-a627-7cfbf36b0751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:58.809906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195d84d-a766-45cf-a627-7cfbf36b0751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:58.813170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195d84d-a766-45cf-a627-7cfbf36b0751 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:58.852577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 873/1767] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 03:59:58.857359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195d84d-a766-45cf-a627-7cfbf36b0751 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:58.899720 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195d84d-a766-45cf-a627-7cfbf36b0751 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:58.917315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf32400-479f-45b1-bf97-eb944fa47fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:58.918237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf32400-479f-45b1-bf97-eb944fa47fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:58.920340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf32400-479f-45b1-bf97-eb944fa47fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:58.921892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 03:59:58.923660 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf32400-479f-45b1-bf97-eb944fa47fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:58.924816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf32400-479f-45b1-bf97-eb944fa47fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:58.927933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf32400-479f-45b1-bf97-eb944fa47fd8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:58.974205 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 874/1768] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 03:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 03:59:58.975652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf32400-479f-45b1-bf97-eb944fa47fd8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:59.004185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af6d97-dbf3-4592-8b7b-8b34bd078123 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:59.008007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf32400-479f-45b1-bf97-eb944fa47fd8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:59.009529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f374fd2-8c8b-4a2d-b37d-f2a17927e2dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:59.010175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f374fd2-8c8b-4a2d-b37d-f2a17927e2dd service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:59.010432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f374fd2-8c8b-4a2d-b37d-f2a17927e2dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:59.010712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f374fd2-8c8b-4a2d-b37d-f2a17927e2dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:59.018510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727c82f-d3a1-4308-ad98-d78fd63d64e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:59.020517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727c82f-d3a1-4308-ad98-d78fd63d64e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:59.021866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727c82f-d3a1-4308-ad98-d78fd63d64e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:59.022396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727c82f-d3a1-4308-ad98-d78fd63d64e2 admin admin] PATH_INFO: `/v3/projects/7e4806116406458e83556bc47ae0a506/users/ce9adf52bd924ccc8e2c994b76914739/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:59.033992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f374fd2-8c8b-4a2d-b37d-f2a17927e2dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:59.033992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f374fd2-8c8b-4a2d-b37d-f2a17927e2dd service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:59.049590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 895/1769] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:59.067181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727c82f-d3a1-4308-ad98-d78fd63d64e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4806116406458e83556bc47ae0a506, user_id=ce9adf52bd924ccc8e2c994b76914739, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:59.070768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727c82f-d3a1-4308-ad98-d78fd63d64e2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:59.100300 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 875/1770] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 03:59:58 2022] PUT /identity/v3/projects/7e4806116406458e83556bc47ae0a506/users/ce9adf52bd924ccc8e2c994b76914739/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 03:59:59.114575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dda62-e091-4d71-9646-7ca21dfacab8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:59.117096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dda62-e091-4d71-9646-7ca21dfacab8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:59.118915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dda62-e091-4d71-9646-7ca21dfacab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:59.180808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-897dda62-e091-4d71-9646-7ca21dfacab8 None None] MFA Rules not processed for user `ce9adf52bd924ccc8e2c994b76914739`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:59.182405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-897dda62-e091-4d71-9646-7ca21dfacab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:59.307828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321fcc3-dc0d-4267-a8e0-b7d087a38e6f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:59.308185 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321fcc3-dc0d-4267-a8e0-b7d087a38e6f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:59.308524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321fcc3-dc0d-4267-a8e0-b7d087a38e6f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:59.311056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 876/1771] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jul 28 03:59:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 03:59:59.368657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 896/1772] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 03:59:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 03:59:59.376733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321fcc3-dc0d-4267-a8e0-b7d087a38e6f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:59.376953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a6fec-ac1c-41de-bd8b-894d7dbe09ae None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:59.377203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a6fec-ac1c-41de-bd8b-894d7dbe09ae None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:59.377573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a6fec-ac1c-41de-bd8b-894d7dbe09ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:59.380324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e321fcc3-dc0d-4267-a8e0-b7d087a38e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:59.410470 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8c7a6fec-ac1c-41de-bd8b-894d7dbe09ae None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 03:59:59.412144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8c7a6fec-ac1c-41de-bd8b-894d7dbe09ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:59.577806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321fcc3-dc0d-4267-a8e0-b7d087a38e6f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:59.579857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 897/1773] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jul 28 03:59:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 03:59:59.595413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a6fec-ac1c-41de-bd8b-894d7dbe09ae None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 03:59:59.598497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8c7a6fec-ac1c-41de-bd8b-894d7dbe09ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 03:59:59.600615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffa6ea-87ce-47fe-8e7e-0cbf23e5b958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:59.601953 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffa6ea-87ce-47fe-8e7e-0cbf23e5b958 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:59.602072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffa6ea-87ce-47fe-8e7e-0cbf23e5b958 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:59.602345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffa6ea-87ce-47fe-8e7e-0cbf23e5b958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:59.614528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffa6ea-87ce-47fe-8e7e-0cbf23e5b958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:59.616457 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffa6ea-87ce-47fe-8e7e-0cbf23e5b958 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:59.618413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a6fec-ac1c-41de-bd8b-894d7dbe09ae None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 03:59:59.626106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27df4e7-76a0-4727-85ab-d1c31ce9db32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 03:59:59.626865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27df4e7-76a0-4727-85ab-d1c31ce9db32 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 03:59:59.627195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27df4e7-76a0-4727-85ab-d1c31ce9db32 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 03:59:59.627416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27df4e7-76a0-4727-85ab-d1c31ce9db32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 03:59:59.630298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 877/1774] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 03:59:59.638897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27df4e7-76a0-4727-85ab-d1c31ce9db32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 03:59:59.640567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27df4e7-76a0-4727-85ab-d1c31ce9db32 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 03:59:59.656588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 898/1775] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 03:59:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:00:00.759718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffa6ea-87ce-47fe-8e7e-0cbf23e5b958 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:00.795180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:00.870100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffa6ea-87ce-47fe-8e7e-0cbf23e5b958 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:00.877886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:00.878828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:00.879706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:00.880815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] PATH_INFO: `/v3/users/ce9adf52bd924ccc8e2c994b76914739` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:00.893029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9adf52bd924ccc8e2c994b76914739)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:00.896622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:01.036091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9adf52bd924ccc8e2c994b76914739'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:01.039897 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9adf52bd924ccc8e2c994b76914739'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:01.058316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9adf52bd924ccc8e2c994b76914739'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:01.093547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9adf52bd924ccc8e2c994b76914739'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:01.104323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9adf52bd924ccc8e2c994b76914739'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:01.114028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 878/1776] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:00 2022] DELETE /identity/v3/users/ce9adf52bd924ccc8e2c994b76914739 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:01.118213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9596-7e57-4be8-ad50-91a59ba48b01 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:01.118751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9596-7e57-4be8-ad50-91a59ba48b01 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:01.118751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9596-7e57-4be8-ad50-91a59ba48b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:01.159793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9c3a9596-7e57-4be8-ad50-91a59ba48b01 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:01.162000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9c3a9596-7e57-4be8-ad50-91a59ba48b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:01.217188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:01.243359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 899/1777] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:01.253660 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:01.255137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:01.268484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d92fc-1136-4db0-add2-b66908b1fc22 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:01.268778 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d92fc-1136-4db0-add2-b66908b1fc22 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:01.269058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d92fc-1136-4db0-add2-b66908b1fc22 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:01.270841 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 900/1778] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jul 28 04:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 04:00:01.288927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0496719-6a8f-484f-9a7d-eb22c72ffe3d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:01.289163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0496719-6a8f-484f-9a7d-eb22c72ffe3d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:01.289453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0496719-6a8f-484f-9a7d-eb22c72ffe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:01.320642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b0496719-6a8f-484f-9a7d-eb22c72ffe3d None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:01.321900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b0496719-6a8f-484f-9a7d-eb22c72ffe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:01.333135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da82225-6a47-480b-9c93-e9fe71061a40 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:01.337137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea078c3-52c7-4560-bcab-bf5fb04d9176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:01.337974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea078c3-52c7-4560-bcab-bf5fb04d9176 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:01.338355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea078c3-52c7-4560-bcab-bf5fb04d9176 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:01.338750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea078c3-52c7-4560-bcab-bf5fb04d9176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:01.350296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea078c3-52c7-4560-bcab-bf5fb04d9176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:01.352547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea078c3-52c7-4560-bcab-bf5fb04d9176 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:01.369681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 879/1779] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:01.427966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 901/1780] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jul 28 04:00:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:01.446954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea078c3-52c7-4560-bcab-bf5fb04d9176 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:01.451585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0ea078c3-52c7-4560-bcab-bf5fb04d9176 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:01.479188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea078c3-52c7-4560-bcab-bf5fb04d9176 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:01.485740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1e01d-60a8-42da-9833-507d91704676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:01.486487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1e01d-60a8-42da-9833-507d91704676 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:01.487172 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1e01d-60a8-42da-9833-507d91704676 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:01.487172 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1e01d-60a8-42da-9833-507d91704676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:01.497492 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1e01d-60a8-42da-9833-507d91704676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:01.500080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1e01d-60a8-42da-9833-507d91704676 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:01.515865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 880/1781] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:00:01.595367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c2b2c-7b48-4111-8c96-83c695cbc6ae None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:01.596800 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c2b2c-7b48-4111-8c96-83c695cbc6ae None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:01.596962 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c2b2c-7b48-4111-8c96-83c695cbc6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:01.653198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-273c2b2c-7b48-4111-8c96-83c695cbc6ae None None] MFA Rules not processed for user `a86c1febcbbe49a0881dfb9334cedba2`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:01.654448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-273c2b2c-7b48-4111-8c96-83c695cbc6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:01.759858 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 902/1782] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:01.764370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1e01d-60a8-42da-9833-507d91704676 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:01.765901 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-47f1e01d-60a8-42da-9833-507d91704676 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:01.800755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c2b2c-7b48-4111-8c96-83c695cbc6ae None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:01.802530 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-273c2b2c-7b48-4111-8c96-83c695cbc6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:01.809904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1e01d-60a8-42da-9833-507d91704676 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:01.817048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad0def-2ebf-4ec9-be8a-c3279806513e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:01.817839 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad0def-2ebf-4ec9-be8a-c3279806513e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:01.818485 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad0def-2ebf-4ec9-be8a-c3279806513e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:01.819008 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad0def-2ebf-4ec9-be8a-c3279806513e admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:01.834496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad0def-2ebf-4ec9-be8a-c3279806513e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:01.837458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c2b2c-7b48-4111-8c96-83c695cbc6ae None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:01.839495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad0def-2ebf-4ec9-be8a-c3279806513e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:01.851958 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:01.852695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:01.852966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:01.853377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] PATH_INFO: `/v3/projects/7e4806116406458e83556bc47ae0a506` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:01.855333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e4806116406458e83556bc47ae0a506)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:01.857444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:01.897978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 881/1783] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:01 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:01.901804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad0def-2ebf-4ec9-be8a-c3279806513e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:01.915162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:01.926381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4806116406458e83556bc47ae0a506', 'request_id': 'req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:01.935619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4806116406458e83556bc47ae0a506', 'request_id': 'req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:01.938354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:01.943599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4806116406458e83556bc47ae0a506', 'request_id': 'req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:01.961246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] The token cache is being invalidate because project 7e4806116406458e83556bc47ae0a506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:01.961678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:01.962632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c1261f7-b5e1-48e1-91cf-820dd2aefe33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:01.964361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 903/1784] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:01 2022] DELETE /identity/v3/projects/7e4806116406458e83556bc47ae0a506 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:01.969432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf48e37-b6d5-42f8-b785-bd31362251bd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:01.969878 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf48e37-b6d5-42f8-b785-bd31362251bd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:01.970561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf48e37-b6d5-42f8-b785-bd31362251bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:01.982315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad0def-2ebf-4ec9-be8a-c3279806513e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:01.991795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cdf48e37-b6d5-42f8-b785-bd31362251bd None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:01.993525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cdf48e37-b6d5-42f8-b785-bd31362251bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:02.048346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40b881-8e79-4fe9-aa06-ea86e0756d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.049271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40b881-8e79-4fe9-aa06-ea86e0756d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.049552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40b881-8e79-4fe9-aa06-ea86e0756d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.049828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40b881-8e79-4fe9-aa06-ea86e0756d37 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.050499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40b881-8e79-4fe9-aa06-ea86e0756d37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.052486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40b881-8e79-4fe9-aa06-ea86e0756d37 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.056212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 904/1785] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:00:02.065789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135671b3-2496-4eeb-b4c7-f588adcbfe2e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.066093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135671b3-2496-4eeb-b4c7-f588adcbfe2e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.066540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135671b3-2496-4eeb-b4c7-f588adcbfe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.077860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 882/1786] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:01 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:02.082539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40b881-8e79-4fe9-aa06-ea86e0756d37 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.084272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4c40b881-8e79-4fe9-aa06-ea86e0756d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:02.088262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-135671b3-2496-4eeb-b4c7-f588adcbfe2e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:02.089530 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-135671b3-2496-4eeb-b4c7-f588adcbfe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:02.097247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40b881-8e79-4fe9-aa06-ea86e0756d37 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.100252 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 905/1787] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:00:02.104551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135671b3-2496-4eeb-b4c7-f588adcbfe2e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.105630 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed7bc2-2193-4935-830a-e4a09530416f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.106307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed7bc2-2193-4935-830a-e4a09530416f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.106587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed7bc2-2193-4935-830a-e4a09530416f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.106889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed7bc2-2193-4935-830a-e4a09530416f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.107604 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed7bc2-2193-4935-830a-e4a09530416f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.109266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed7bc2-2193-4935-830a-e4a09530416f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.116901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135671b3-2496-4eeb-b4c7-f588adcbfe2e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.121131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 883/1788] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:00:02.124029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f06673-1bf0-4869-8547-464839410b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.124707 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f06673-1bf0-4869-8547-464839410b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.124989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f06673-1bf0-4869-8547-464839410b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.125253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f06673-1bf0-4869-8547-464839410b3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.126555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f06673-1bf0-4869-8547-464839410b3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:00:02.127192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f06673-1bf0-4869-8547-464839410b3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.127996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed7bc2-2193-4935-830a-e4a09530416f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.128526 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f06673-1bf0-4869-8547-464839410b3f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.139797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed7bc2-2193-4935-830a-e4a09530416f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.143284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 906/1789] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:00:02.147851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f06673-1bf0-4869-8547-464839410b3f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.149021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59537d7-78ba-4b41-bb03-b043f5880e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.149733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59537d7-78ba-4b41-bb03-b043f5880e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.150034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59537d7-78ba-4b41-bb03-b043f5880e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.150312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59537d7-78ba-4b41-bb03-b043f5880e2a admin admin] PATH_INFO: `/v3/projects/12b2e384a4bf49628b6c01abcdd6e66b/users/5ad5416bce8f4390988d0ce1300181d1/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.159992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f06673-1bf0-4869-8547-464839410b3f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.166160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfb112-558d-4efa-8eb2-2a5795457f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.166427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfb112-558d-4efa-8eb2-2a5795457f49 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.166712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfb112-558d-4efa-8eb2-2a5795457f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.167049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfb112-558d-4efa-8eb2-2a5795457f49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.167738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfb112-558d-4efa-8eb2-2a5795457f49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.169701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfb112-558d-4efa-8eb2-2a5795457f49 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.170253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59537d7-78ba-4b41-bb03-b043f5880e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b2e384a4bf49628b6c01abcdd6e66b, user_id=5ad5416bce8f4390988d0ce1300181d1, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.173689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59537d7-78ba-4b41-bb03-b043f5880e2a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.191825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 884/1790] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:02 2022] PUT /identity/v3/projects/12b2e384a4bf49628b6c01abcdd6e66b/users/5ad5416bce8f4390988d0ce1300181d1/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:02.197524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59537d7-78ba-4b41-bb03-b043f5880e2a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.207285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 907/1791] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:02 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:00:02.208255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfb112-558d-4efa-8eb2-2a5795457f49 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.267237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59537d7-78ba-4b41-bb03-b043f5880e2a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.271811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bc596-1b60-4253-844d-888c85a0e69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.272543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bc596-1b60-4253-844d-888c85a0e69f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.272837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bc596-1b60-4253-844d-888c85a0e69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.273105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bc596-1b60-4253-844d-888c85a0e69f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.273802 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bc596-1b60-4253-844d-888c85a0e69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.275429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bc596-1b60-4253-844d-888c85a0e69f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.276527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfb112-558d-4efa-8eb2-2a5795457f49 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.282459 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c615001-8246-4dad-8fa1-2d11d1de4bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.283103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c615001-8246-4dad-8fa1-2d11d1de4bff admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.283387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c615001-8246-4dad-8fa1-2d11d1de4bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.283655 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c615001-8246-4dad-8fa1-2d11d1de4bff admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.284318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c615001-8246-4dad-8fa1-2d11d1de4bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.286261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c615001-8246-4dad-8fa1-2d11d1de4bff admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.290982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 885/1792] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:00:02.295027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bc596-1b60-4253-844d-888c85a0e69f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.302865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:02.308069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bc596-1b60-4253-844d-888c85a0e69f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.315331 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e0ac4-a265-460b-a5de-f1758602101c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.316225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e0ac4-a265-460b-a5de-f1758602101c admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.316635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e0ac4-a265-460b-a5de-f1758602101c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.316944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 908/1793] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:02 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:02.317107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e0ac4-a265-460b-a5de-f1758602101c admin admin] PATH_INFO: `/v3/projects/12b2e384a4bf49628b6c01abcdd6e66b/users/5ad5416bce8f4390988d0ce1300181d1/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.319505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e0ac4-a265-460b-a5de-f1758602101c admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b2e384a4bf49628b6c01abcdd6e66b, user_id=5ad5416bce8f4390988d0ce1300181d1, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.320971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c615001-8246-4dad-8fa1-2d11d1de4bff admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.322182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e0ac4-a265-460b-a5de-f1758602101c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.333291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 886/1794] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:02 2022] PUT /identity/v3/projects/12b2e384a4bf49628b6c01abcdd6e66b/users/5ad5416bce8f4390988d0ce1300181d1/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:02.339246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c615001-8246-4dad-8fa1-2d11d1de4bff admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.399226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:02.437594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7bc68-8f33-4366-8dd0-2f9d3e563a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.438269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7bc68-8f33-4366-8dd0-2f9d3e563a89 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.438561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7bc68-8f33-4366-8dd0-2f9d3e563a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.438860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7bc68-8f33-4366-8dd0-2f9d3e563a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.439781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7bc68-8f33-4366-8dd0-2f9d3e563a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.441844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7bc68-8f33-4366-8dd0-2f9d3e563a89 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.458430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 909/1795] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:00:02.459770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e0ac4-a265-460b-a5de-f1758602101c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.472235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e0ac4-a265-460b-a5de-f1758602101c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.480402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35cfad-7f50-47d8-8cf1-55ee9b1d6043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.481082 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35cfad-7f50-47d8-8cf1-55ee9b1d6043 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.481451 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35cfad-7f50-47d8-8cf1-55ee9b1d6043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.481713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35cfad-7f50-47d8-8cf1-55ee9b1d6043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.482458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35cfad-7f50-47d8-8cf1-55ee9b1d6043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.484175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35cfad-7f50-47d8-8cf1-55ee9b1d6043 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.499577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 887/1796] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:00:02.501524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7bc68-8f33-4366-8dd0-2f9d3e563a89 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.512801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7bc68-8f33-4366-8dd0-2f9d3e563a89 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.518033 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d85d2-931f-498e-961d-04f9646a97d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.518918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d85d2-931f-498e-961d-04f9646a97d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.519203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d85d2-931f-498e-961d-04f9646a97d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.519478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d85d2-931f-498e-961d-04f9646a97d9 admin admin] PATH_INFO: `/v3/projects/7c005adc61db4faf8803b57f819e2461/users/4ed870efee404214b9f4691f3fc0eb3a/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.542251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d85d2-931f-498e-961d-04f9646a97d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c005adc61db4faf8803b57f819e2461, user_id=4ed870efee404214b9f4691f3fc0eb3a, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.545195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d85d2-931f-498e-961d-04f9646a97d9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.567262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 910/1797] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:02 2022] PUT /identity/v3/projects/7c005adc61db4faf8803b57f819e2461/users/4ed870efee404214b9f4691f3fc0eb3a/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:02.578075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1638ebc-43e3-4c05-9dcd-956c89d4ea63 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.578513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1638ebc-43e3-4c05-9dcd-956c89d4ea63 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.579009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1638ebc-43e3-4c05-9dcd-956c89d4ea63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.614260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c1638ebc-43e3-4c05-9dcd-956c89d4ea63 None None] MFA Rules not processed for user `4ed870efee404214b9f4691f3fc0eb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:02.617029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c1638ebc-43e3-4c05-9dcd-956c89d4ea63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:02.731010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 888/1798] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:00:02.731098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d85d2-931f-498e-961d-04f9646a97d9 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.812139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d85d2-931f-498e-961d-04f9646a97d9 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.817971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7237308-085b-4dab-9cd6-3b4c70937a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.818538 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7237308-085b-4dab-9cd6-3b4c70937a39 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.819114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7237308-085b-4dab-9cd6-3b4c70937a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.819114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7237308-085b-4dab-9cd6-3b4c70937a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.819786 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7237308-085b-4dab-9cd6-3b4c70937a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.821829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7237308-085b-4dab-9cd6-3b4c70937a39 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.852832 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 911/1799] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:02 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:02.857081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1638ebc-43e3-4c05-9dcd-956c89d4ea63 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.871137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1638ebc-43e3-4c05-9dcd-956c89d4ea63 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.940289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80535e-b329-476e-a79b-4aaf0196caa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.941214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80535e-b329-476e-a79b-4aaf0196caa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.941505 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80535e-b329-476e-a79b-4aaf0196caa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.941782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80535e-b329-476e-a79b-4aaf0196caa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.943002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80535e-b329-476e-a79b-4aaf0196caa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.944932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80535e-b329-476e-a79b-4aaf0196caa9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:02.978425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 889/1800] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:02 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:02.979643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7237308-085b-4dab-9cd6-3b4c70937a39 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:02.991221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7237308-085b-4dab-9cd6-3b4c70937a39 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:02.995911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15308a-872d-4d09-9c06-49807e098678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:02.996532 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15308a-872d-4d09-9c06-49807e098678 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:02.996816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15308a-872d-4d09-9c06-49807e098678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:02.997208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15308a-872d-4d09-9c06-49807e098678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:02.997921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15308a-872d-4d09-9c06-49807e098678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:02.999748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15308a-872d-4d09-9c06-49807e098678 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:03.014390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 912/1801] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:03.018936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80535e-b329-476e-a79b-4aaf0196caa9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:03.032842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80535e-b329-476e-a79b-4aaf0196caa9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:03.037826 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db62dfd-01f8-4add-8608-cc64dac45974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:03.038845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db62dfd-01f8-4add-8608-cc64dac45974 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:03.039141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db62dfd-01f8-4add-8608-cc64dac45974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:03.039424 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db62dfd-01f8-4add-8608-cc64dac45974 admin admin] PATH_INFO: `/v3/projects/8197e5897bdf489f936cfe5e07456fec/users/8e58adb7ee704d53ba8a080927771c08/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:03.060424 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db62dfd-01f8-4add-8608-cc64dac45974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8197e5897bdf489f936cfe5e07456fec, user_id=8e58adb7ee704d53ba8a080927771c08, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:03.063105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db62dfd-01f8-4add-8608-cc64dac45974 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:03.087413 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 890/1802] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:03 2022] PUT /identity/v3/projects/8197e5897bdf489f936cfe5e07456fec/users/8e58adb7ee704d53ba8a080927771c08/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:03.088837 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15308a-872d-4d09-9c06-49807e098678 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:03.101452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15308a-872d-4d09-9c06-49807e098678 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:03.182630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a70a04-23fb-421f-a172-d4a1aa62fc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:03.183355 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a70a04-23fb-421f-a172-d4a1aa62fc6e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:03.183558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a70a04-23fb-421f-a172-d4a1aa62fc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:03.183891 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a70a04-23fb-421f-a172-d4a1aa62fc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:03.184592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a70a04-23fb-421f-a172-d4a1aa62fc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:03.191011 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a70a04-23fb-421f-a172-d4a1aa62fc6e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:03.202130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 913/1803] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:03.207602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db62dfd-01f8-4add-8608-cc64dac45974 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:03.219438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db62dfd-01f8-4add-8608-cc64dac45974 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:03.225223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1e361-1175-4ab3-a691-fc26af5ce722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:03.225917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e361-1175-4ab3-a691-fc26af5ce722 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:03.226206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e361-1175-4ab3-a691-fc26af5ce722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:03.226486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e361-1175-4ab3-a691-fc26af5ce722 admin admin] PATH_INFO: `/v3/projects/8197e5897bdf489f936cfe5e07456fec/users/8e58adb7ee704d53ba8a080927771c08/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:03.228705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1e361-1175-4ab3-a691-fc26af5ce722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8197e5897bdf489f936cfe5e07456fec, user_id=8e58adb7ee704d53ba8a080927771c08, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:03.231425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1e361-1175-4ab3-a691-fc26af5ce722 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:03.251600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 891/1804] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:03 2022] PUT /identity/v3/projects/8197e5897bdf489f936cfe5e07456fec/users/8e58adb7ee704d53ba8a080927771c08/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:03.255390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ecd0b-e2d5-4280-bf13-b1ced90b5e99 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:03.255678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ecd0b-e2d5-4280-bf13-b1ced90b5e99 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:03.255966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ecd0b-e2d5-4280-bf13-b1ced90b5e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:03.286496 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-797ecd0b-e2d5-4280-bf13-b1ced90b5e99 None None] MFA Rules not processed for user `8e58adb7ee704d53ba8a080927771c08`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:03.287742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-797ecd0b-e2d5-4280-bf13-b1ced90b5e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:03.296922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:03.369498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 914/1805] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:03.383227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1e361-1175-4ab3-a691-fc26af5ce722 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:03.384960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e5a1e361-1175-4ab3-a691-fc26af5ce722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:03.406568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974408-4e5d-45af-b6d9-3040c1ce1869 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:03.406937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974408-4e5d-45af-b6d9-3040c1ce1869 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:03.407827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974408-4e5d-45af-b6d9-3040c1ce1869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:03.428803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:03.442314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e2974408-4e5d-45af-b6d9-3040c1ce1869 None None] MFA Rules not processed for user `5ad5416bce8f4390988d0ce1300181d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:03.443689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e2974408-4e5d-45af-b6d9-3040c1ce1869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:03.464526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1e361-1175-4ab3-a691-fc26af5ce722 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:03.471051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f7165-db1b-4e7e-9fd0-14f5174da0cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:03.471698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f7165-db1b-4e7e-9fd0-14f5174da0cb service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:03.471985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f7165-db1b-4e7e-9fd0-14f5174da0cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:03.472264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f7165-db1b-4e7e-9fd0-14f5174da0cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:03.484064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f7165-db1b-4e7e-9fd0-14f5174da0cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:03.486063 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f7165-db1b-4e7e-9fd0-14f5174da0cb service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:03.499670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 892/1806] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:03.537379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 915/1807] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:00:03.572114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f7165-db1b-4e7e-9fd0-14f5174da0cb service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:03.574842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3f8f7165-db1b-4e7e-9fd0-14f5174da0cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:03.597721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f7165-db1b-4e7e-9fd0-14f5174da0cb service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:03.615509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65a400-8078-4282-a193-34ed102ae1c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:03.617717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65a400-8078-4282-a193-34ed102ae1c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:03.618272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65a400-8078-4282-a193-34ed102ae1c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:03.618495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65a400-8078-4282-a193-34ed102ae1c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:03.624031 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ba65a400-8078-4282-a193-34ed102ae1c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:03.627505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2974408-4e5d-45af-b6d9-3040c1ce1869 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:03.644432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2974408-4e5d-45af-b6d9-3040c1ce1869 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:03.649686 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edefd7-35d8-4a60-91ab-12cfa8026ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:03.650611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edefd7-35d8-4a60-91ab-12cfa8026ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:03.651033 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edefd7-35d8-4a60-91ab-12cfa8026ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:03.651396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edefd7-35d8-4a60-91ab-12cfa8026ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:03.713252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65a400-8078-4282-a193-34ed102ae1c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:03.715301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65a400-8078-4282-a193-34ed102ae1c3 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:03.727917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 893/1808] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:00:03.765818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edefd7-35d8-4a60-91ab-12cfa8026ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:03.768110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edefd7-35d8-4a60-91ab-12cfa8026ce1 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:03.793340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 916/1809] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:05.222034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65a400-8078-4282-a193-34ed102ae1c3 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:05.224649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ba65a400-8078-4282-a193-34ed102ae1c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:05.347217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65a400-8078-4282-a193-34ed102ae1c3 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:05.352153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:05.352838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:05.353146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:05.353426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] PATH_INFO: `/v3/users/0609476b0b494fc69dd159b7112136cc` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:05.354602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] RBAC: Authorizing `identity:delete_user(user_id=0609476b0b494fc69dd159b7112136cc)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:05.356715 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:05.418958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0609476b0b494fc69dd159b7112136cc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:05.419832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0609476b0b494fc69dd159b7112136cc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:05.431404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0609476b0b494fc69dd159b7112136cc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:05.441504 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:05.446809 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0609476b0b494fc69dd159b7112136cc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:05.459465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0609476b0b494fc69dd159b7112136cc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:05.466827 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 894/1810] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:05 2022] DELETE /identity/v3/users/0609476b0b494fc69dd159b7112136cc => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:05.472352 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e9296-d279-4e6a-9aef-7a789c7c1b49 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:05.472910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e9296-d279-4e6a-9aef-7a789c7c1b49 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:05.473049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e9296-d279-4e6a-9aef-7a789c7c1b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:05.504276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-943e9296-d279-4e6a-9aef-7a789c7c1b49 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:05.505584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-943e9296-d279-4e6a-9aef-7a789c7c1b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:05.544090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:05.586015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:05.587539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:05.615076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 917/1811] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:00:05.624996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e9296-d279-4e6a-9aef-7a789c7c1b49 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:05.628240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-943e9296-d279-4e6a-9aef-7a789c7c1b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:05.741205 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8f4e0-31ba-4b91-a42d-46cdcba449da admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:05.749099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a15bb-9dfe-4d31-b5ce-4dc6fc7e439f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:05.749887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a15bb-9dfe-4d31-b5ce-4dc6fc7e439f service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:05.750306 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a15bb-9dfe-4d31-b5ce-4dc6fc7e439f service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:05.752019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a15bb-9dfe-4d31-b5ce-4dc6fc7e439f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:05.756640 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4d0a15bb-9dfe-4d31-b5ce-4dc6fc7e439f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:05.764589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e9296-d279-4e6a-9aef-7a789c7c1b49 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:05.780391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0836757-0ba5-4c9b-9bb0-8688a31700b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:05.780976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0836757-0ba5-4c9b-9bb0-8688a31700b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:05.781257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0836757-0ba5-4c9b-9bb0-8688a31700b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:05.781529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0836757-0ba5-4c9b-9bb0-8688a31700b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:05.799450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0836757-0ba5-4c9b-9bb0-8688a31700b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:05.800909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0836757-0ba5-4c9b-9bb0-8688a31700b6 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:05.820885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 918/1812] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:05.862730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a15bb-9dfe-4d31-b5ce-4dc6fc7e439f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:05.873333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a15bb-9dfe-4d31-b5ce-4dc6fc7e439f service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:05.895625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 895/1813] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:05 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:00:06.388262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0836757-0ba5-4c9b-9bb0-8688a31700b6 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:06.390012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c0836757-0ba5-4c9b-9bb0-8688a31700b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:06.432645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0836757-0ba5-4c9b-9bb0-8688a31700b6 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:06.457927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:06.459622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:06.460136 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:06.465178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] PATH_INFO: `/v3/projects/f80df879907e4c94b2b55431a434417d` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:06.467166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] RBAC: Authorizing `identity:delete_project(project_id=f80df879907e4c94b2b55431a434417d)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:06.470041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:06.635631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f80df879907e4c94b2b55431a434417d', 'request_id': 'req-9995d658-accd-4b86-84dc-92772767f7da'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:06.643002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f80df879907e4c94b2b55431a434417d', 'request_id': 'req-9995d658-accd-4b86-84dc-92772767f7da'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:06.655785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f80df879907e4c94b2b55431a434417d', 'request_id': 'req-9995d658-accd-4b86-84dc-92772767f7da'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:06.691828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] The token cache is being invalidate because project f80df879907e4c94b2b55431a434417d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:06.692285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:06.693787 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:06.699972 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 919/1814] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:06 2022] DELETE /identity/v3/projects/f80df879907e4c94b2b55431a434417d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:06.707325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a15bb-9dfe-4d31-b5ce-4dc6fc7e439f service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:06.721858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a15bb-9dfe-4d31-b5ce-4dc6fc7e439f service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:06.946139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffeb61-5b81-44c1-994b-564c87e4a56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:06.947149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffeb61-5b81-44c1-994b-564c87e4a56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:06.947595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffeb61-5b81-44c1-994b-564c87e4a56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:06.948036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffeb61-5b81-44c1-994b-564c87e4a56c admin admin] PATH_INFO: `/v3/users/f751b14f54224be492c205f228394365` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:06.952139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffeb61-5b81-44c1-994b-564c87e4a56c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f751b14f54224be492c205f228394365)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:06.955075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffeb61-5b81-44c1-994b-564c87e4a56c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:07.076140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:07.118821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-73ffeb61-5b81-44c1-994b-564c87e4a56c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f751b14f54224be492c205f228394365'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:07.118821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-73ffeb61-5b81-44c1-994b-564c87e4a56c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f751b14f54224be492c205f228394365'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:07.125872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-73ffeb61-5b81-44c1-994b-564c87e4a56c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f751b14f54224be492c205f228394365'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:07.140381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-73ffeb61-5b81-44c1-994b-564c87e4a56c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f751b14f54224be492c205f228394365'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:07.150215 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-73ffeb61-5b81-44c1-994b-564c87e4a56c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f751b14f54224be492c205f228394365'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:07.158807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 896/1815] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:06 2022] DELETE /identity/v3/users/f751b14f54224be492c205f228394365 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:07.523164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:07.524586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:07.577008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995d658-accd-4b86-84dc-92772767f7da admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:07.643250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:07.659637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:07.660534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:07.660803 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:07.661134 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] PATH_INFO: `/v3/projects/6cf630b7ba0b43bfaad2647f706fbf17` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:07.662601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cf630b7ba0b43bfaad2647f706fbf17)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:07.665035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:07.727690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cf630b7ba0b43bfaad2647f706fbf17', 'request_id': 'req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:07.733663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cf630b7ba0b43bfaad2647f706fbf17', 'request_id': 'req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:07.741128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cf630b7ba0b43bfaad2647f706fbf17', 'request_id': 'req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:07.760388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] The token cache is being invalidate because project 6cf630b7ba0b43bfaad2647f706fbf17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:07.760814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:07.761696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-059f5c2f-c4e4-4a15-a75e-dd5f27ffde00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:07.766826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 920/1816] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:07 2022] DELETE /identity/v3/projects/6cf630b7ba0b43bfaad2647f706fbf17 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:07.769397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b36011-46d7-4d31-a4bc-35866529db19 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:07.769889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b36011-46d7-4d31-a4bc-35866529db19 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:07.769976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b36011-46d7-4d31-a4bc-35866529db19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:07.791845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-32b36011-46d7-4d31-a4bc-35866529db19 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:07.793331 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-32b36011-46d7-4d31-a4bc-35866529db19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:07.861536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 897/1817] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:07.870668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f749fd9-3784-4eae-872f-a6d1c4583d90 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:07.871008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f749fd9-3784-4eae-872f-a6d1c4583d90 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:07.871291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f749fd9-3784-4eae-872f-a6d1c4583d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:07.892211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1f749fd9-3784-4eae-872f-a6d1c4583d90 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:07.893479 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1f749fd9-3784-4eae-872f-a6d1c4583d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:07.901554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 921/1818] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:07.905799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b36011-46d7-4d31-a4bc-35866529db19 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:07.916572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b36011-46d7-4d31-a4bc-35866529db19 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:07.921118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695337be-1a18-46b3-95da-966756c7e979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:07.921734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695337be-1a18-46b3-95da-966756c7e979 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:07.922015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695337be-1a18-46b3-95da-966756c7e979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:07.922290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695337be-1a18-46b3-95da-966756c7e979 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:07.923976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695337be-1a18-46b3-95da-966756c7e979 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:00:07.924287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695337be-1a18-46b3-95da-966756c7e979 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:07.925878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695337be-1a18-46b3-95da-966756c7e979 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:07.942095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 898/1819] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:00:07.946297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f749fd9-3784-4eae-872f-a6d1c4583d90 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:07.955615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f749fd9-3784-4eae-872f-a6d1c4583d90 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:07.960265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e1705-b74a-4893-bbe5-5116a12c3043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:07.960868 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1705-b74a-4893-bbe5-5116a12c3043 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:07.961152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1705-b74a-4893-bbe5-5116a12c3043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:07.961434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1705-b74a-4893-bbe5-5116a12c3043 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:07.962119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e1705-b74a-4893-bbe5-5116a12c3043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:07.964061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e1705-b74a-4893-bbe5-5116a12c3043 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:07.991563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 922/1820] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:07 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:07.995661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695337be-1a18-46b3-95da-966756c7e979 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:08.006102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695337be-1a18-46b3-95da-966756c7e979 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:08.061376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592a516-1854-47aa-b1da-263081e59ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:08.061985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592a516-1854-47aa-b1da-263081e59ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.062268 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592a516-1854-47aa-b1da-263081e59ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.062540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592a516-1854-47aa-b1da-263081e59ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.063352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592a516-1854-47aa-b1da-263081e59ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:08.065289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592a516-1854-47aa-b1da-263081e59ab0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:08.089405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 899/1821] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:07 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:08.090577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e1705-b74a-4893-bbe5-5116a12c3043 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:08.101693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e1705-b74a-4893-bbe5-5116a12c3043 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:08.106418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9397d-544d-46c7-a338-7bb337466eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:08.107069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9397d-544d-46c7-a338-7bb337466eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.107364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9397d-544d-46c7-a338-7bb337466eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.107642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9397d-544d-46c7-a338-7bb337466eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.108351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9397d-544d-46c7-a338-7bb337466eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:08.109997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9397d-544d-46c7-a338-7bb337466eb2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:08.121067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 923/1822] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:08.125422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592a516-1854-47aa-b1da-263081e59ab0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:08.136489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592a516-1854-47aa-b1da-263081e59ab0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:08.150262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a8058-6033-405d-9f92-e3c000b45b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:08.151037 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a8058-6033-405d-9f92-e3c000b45b3c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.151428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a8058-6033-405d-9f92-e3c000b45b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.151811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a8058-6033-405d-9f92-e3c000b45b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.152683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a8058-6033-405d-9f92-e3c000b45b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:08.154418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a8058-6033-405d-9f92-e3c000b45b3c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:08.168925 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 900/1823] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:08.173982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9397d-544d-46c7-a338-7bb337466eb2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:08.184236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9397d-544d-46c7-a338-7bb337466eb2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:08.189146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01ba26-e210-4600-9182-f0cda3bd268d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:08.189912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01ba26-e210-4600-9182-f0cda3bd268d admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.190294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01ba26-e210-4600-9182-f0cda3bd268d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.193231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01ba26-e210-4600-9182-f0cda3bd268d admin admin] PATH_INFO: `/v3/projects/75c305d314c54bccbd441bf235d23194/users/a9446a907678458394c6a29bcc541e02/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.205889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01ba26-e210-4600-9182-f0cda3bd268d admin admin] RBAC: Authorizing `identity:create_grant(project_id=75c305d314c54bccbd441bf235d23194, user_id=a9446a907678458394c6a29bcc541e02, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:08.208705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01ba26-e210-4600-9182-f0cda3bd268d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:08.226365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 924/1824] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:08 2022] PUT /identity/v3/projects/75c305d314c54bccbd441bf235d23194/users/a9446a907678458394c6a29bcc541e02/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:08.237558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0b5ca-767b-4d01-8059-49f5e2b77125 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.237879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0b5ca-767b-4d01-8059-49f5e2b77125 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.238159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0b5ca-767b-4d01-8059-49f5e2b77125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.272394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d3d0b5ca-767b-4d01-8059-49f5e2b77125 None None] MFA Rules not processed for user `a9446a907678458394c6a29bcc541e02`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:08.274367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d3d0b5ca-767b-4d01-8059-49f5e2b77125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:08.279932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:08.357030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 901/1825] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:08.362187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01ba26-e210-4600-9182-f0cda3bd268d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:08.425484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01ba26-e210-4600-9182-f0cda3bd268d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:08.430264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a705c4-1467-4978-9e49-e7ce59a7d726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:08.430909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a705c4-1467-4978-9e49-e7ce59a7d726 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.431193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a705c4-1467-4978-9e49-e7ce59a7d726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.431468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a705c4-1467-4978-9e49-e7ce59a7d726 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.432141 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a705c4-1467-4978-9e49-e7ce59a7d726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:08.434037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a705c4-1467-4978-9e49-e7ce59a7d726 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:08.462828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 925/1826] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:08 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:00:08.465268 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0b5ca-767b-4d01-8059-49f5e2b77125 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:08.476632 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0b5ca-767b-4d01-8059-49f5e2b77125 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:08.532649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b828b-28a6-47bd-bb25-7fb2dd311add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:08.533380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b828b-28a6-47bd-bb25-7fb2dd311add admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.533782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b828b-28a6-47bd-bb25-7fb2dd311add admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.534162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b828b-28a6-47bd-bb25-7fb2dd311add admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.534965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b828b-28a6-47bd-bb25-7fb2dd311add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:08.537204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b828b-28a6-47bd-bb25-7fb2dd311add admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:08.560952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 902/1827] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:08 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:08.564882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a705c4-1467-4978-9e49-e7ce59a7d726 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:08.575047 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a705c4-1467-4978-9e49-e7ce59a7d726 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:08.580127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cf050-04d8-47d9-861b-40743d181e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:08.580724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cf050-04d8-47d9-861b-40743d181e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.581001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cf050-04d8-47d9-861b-40743d181e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.581276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cf050-04d8-47d9-861b-40743d181e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.581967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cf050-04d8-47d9-861b-40743d181e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:08.583612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cf050-04d8-47d9-861b-40743d181e67 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:08.584820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:08.596581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 926/1828] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:08.600613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b828b-28a6-47bd-bb25-7fb2dd311add admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:08.611579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b828b-28a6-47bd-bb25-7fb2dd311add admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:08.616167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa71f-2ccb-4db8-98f7-16b003adf488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:08.616830 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa71f-2ccb-4db8-98f7-16b003adf488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.617114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa71f-2ccb-4db8-98f7-16b003adf488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.617526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa71f-2ccb-4db8-98f7-16b003adf488 admin admin] PATH_INFO: `/v3/projects/7471b564de914bd9bdc1208750075fe8/users/c696ac1050864f77a0dc8a5adae70a4e/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.634650 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aa71f-2ccb-4db8-98f7-16b003adf488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7471b564de914bd9bdc1208750075fe8, user_id=c696ac1050864f77a0dc8a5adae70a4e, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:08.637730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aa71f-2ccb-4db8-98f7-16b003adf488 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:08.651396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 903/1829] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:08 2022] PUT /identity/v3/projects/7471b564de914bd9bdc1208750075fe8/users/c696ac1050864f77a0dc8a5adae70a4e/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:08.655362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cf050-04d8-47d9-861b-40743d181e67 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:08.665897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cf050-04d8-47d9-861b-40743d181e67 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:08.721842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411663c-7023-4ccb-84eb-1a412b3eec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:08.722450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411663c-7023-4ccb-84eb-1a412b3eec69 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.722733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411663c-7023-4ccb-84eb-1a412b3eec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.723036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411663c-7023-4ccb-84eb-1a412b3eec69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.723723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411663c-7023-4ccb-84eb-1a412b3eec69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:08.725313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411663c-7023-4ccb-84eb-1a412b3eec69 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:08.734701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 927/1830] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:08.739384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa71f-2ccb-4db8-98f7-16b003adf488 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:08.749574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa71f-2ccb-4db8-98f7-16b003adf488 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:08.754201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d8fec-619b-4cc9-b0e7-7e36e249f02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:08.754896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8fec-619b-4cc9-b0e7-7e36e249f02c admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.755179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8fec-619b-4cc9-b0e7-7e36e249f02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.755456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8fec-619b-4cc9-b0e7-7e36e249f02c admin admin] PATH_INFO: `/v3/projects/7471b564de914bd9bdc1208750075fe8/users/c696ac1050864f77a0dc8a5adae70a4e/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.757411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d8fec-619b-4cc9-b0e7-7e36e249f02c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7471b564de914bd9bdc1208750075fe8, user_id=c696ac1050864f77a0dc8a5adae70a4e, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:08.760082 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d8fec-619b-4cc9-b0e7-7e36e249f02c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:08.770755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 904/1831] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:08 2022] PUT /identity/v3/projects/7471b564de914bd9bdc1208750075fe8/users/c696ac1050864f77a0dc8a5adae70a4e/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:08.787703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c07f75-ce79-4064-884c-0ba286216aa3 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.788252 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c07f75-ce79-4064-884c-0ba286216aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.788544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c07f75-ce79-4064-884c-0ba286216aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.818182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d4c07f75-ce79-4064-884c-0ba286216aa3 None None] MFA Rules not processed for user `c696ac1050864f77a0dc8a5adae70a4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:08.819386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d4c07f75-ce79-4064-884c-0ba286216aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:08.887034 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 928/1832] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:08.899076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d8fec-619b-4cc9-b0e7-7e36e249f02c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:08.900770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-419d8fec-619b-4cc9-b0e7-7e36e249f02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:08.959360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d8fec-619b-4cc9-b0e7-7e36e249f02c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:08.963047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586335ce-bb96-4d79-ae0a-06975b3fc94c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:08.963743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586335ce-bb96-4d79-ae0a-06975b3fc94c service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:08.964026 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586335ce-bb96-4d79-ae0a-06975b3fc94c service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:08.964302 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586335ce-bb96-4d79-ae0a-06975b3fc94c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:08.973862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586335ce-bb96-4d79-ae0a-06975b3fc94c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:08.975316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586335ce-bb96-4d79-ae0a-06975b3fc94c service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:08.984915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 905/1833] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:09.013628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c07f75-ce79-4064-884c-0ba286216aa3 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:09.078886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c07f75-ce79-4064-884c-0ba286216aa3 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:09.083424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:09.084074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:09.084359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:09.084650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] PATH_INFO: `/v3/users/a9446a907678458394c6a29bcc541e02` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:09.085824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9446a907678458394c6a29bcc541e02)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:09.087902 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:09.109387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:09.138302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9446a907678458394c6a29bcc541e02'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:09.139009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9446a907678458394c6a29bcc541e02'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:09.151509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9446a907678458394c6a29bcc541e02'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:09.159339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9446a907678458394c6a29bcc541e02'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:09.168101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9446a907678458394c6a29bcc541e02'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:09.174216 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 929/1834] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:09 2022] DELETE /identity/v3/users/a9446a907678458394c6a29bcc541e02 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:09.178729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0c81-2abf-46fb-a68a-9001c0846ed2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:09.179055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0c81-2abf-46fb-a68a-9001c0846ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:09.179337 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0c81-2abf-46fb-a68a-9001c0846ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:09.186353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:09.199487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-60ea0c81-2abf-46fb-a68a-9001c0846ed2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:09.200696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-60ea0c81-2abf-46fb-a68a-9001c0846ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:09.261038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 906/1835] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:09.271663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:09.273073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:09.333437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9cb7b-08bd-4367-85ab-82629ddca4ed admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:09.336816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877310e-8814-4136-a835-bdea6c6b18af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:09.337439 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877310e-8814-4136-a835-bdea6c6b18af service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:09.337726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877310e-8814-4136-a835-bdea6c6b18af service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:09.338010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877310e-8814-4136-a835-bdea6c6b18af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:09.348437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877310e-8814-4136-a835-bdea6c6b18af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:09.349879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877310e-8814-4136-a835-bdea6c6b18af service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:09.361187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 930/1836] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:09.989679 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea0c81-2abf-46fb-a68a-9001c0846ed2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:09.991955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-60ea0c81-2abf-46fb-a68a-9001c0846ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:10.024434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea0c81-2abf-46fb-a68a-9001c0846ed2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:10.039595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:10.040276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:10.040609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:10.041002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] PATH_INFO: `/v3/projects/75c305d314c54bccbd441bf235d23194` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:10.042974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75c305d314c54bccbd441bf235d23194)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:10.045232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:10.161592 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75c305d314c54bccbd441bf235d23194', 'request_id': 'req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:10.183940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75c305d314c54bccbd441bf235d23194', 'request_id': 'req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:10.198306 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75c305d314c54bccbd441bf235d23194', 'request_id': 'req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:10.225844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] The token cache is being invalidate because project 75c305d314c54bccbd441bf235d23194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:10.226195 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:10.226993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:10.228802 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 907/1837] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:09 2022] DELETE /identity/v3/projects/75c305d314c54bccbd441bf235d23194 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:10.233346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877310e-8814-4136-a835-bdea6c6b18af service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:10.244315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877310e-8814-4136-a835-bdea6c6b18af service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:10.335921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1f50b-7f95-4da0-b9e3-8af4b905425f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:10.336517 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1f50b-7f95-4da0-b9e3-8af4b905425f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:10.336824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1f50b-7f95-4da0-b9e3-8af4b905425f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:10.337323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1f50b-7f95-4da0-b9e3-8af4b905425f admin admin] PATH_INFO: `/v3/users/c696ac1050864f77a0dc8a5adae70a4e` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:10.338531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1f50b-7f95-4da0-b9e3-8af4b905425f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c696ac1050864f77a0dc8a5adae70a4e)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:10.340751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1f50b-7f95-4da0-b9e3-8af4b905425f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:10.406285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:10.417709 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e1d1f50b-7f95-4da0-b9e3-8af4b905425f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c696ac1050864f77a0dc8a5adae70a4e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:10.418866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e1d1f50b-7f95-4da0-b9e3-8af4b905425f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c696ac1050864f77a0dc8a5adae70a4e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:10.444223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e1d1f50b-7f95-4da0-b9e3-8af4b905425f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c696ac1050864f77a0dc8a5adae70a4e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:10.452606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e1d1f50b-7f95-4da0-b9e3-8af4b905425f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c696ac1050864f77a0dc8a5adae70a4e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:10.463750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e1d1f50b-7f95-4da0-b9e3-8af4b905425f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c696ac1050864f77a0dc8a5adae70a4e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:10.476980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 931/1838] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:10 2022] DELETE /identity/v3/users/c696ac1050864f77a0dc8a5adae70a4e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:12.160037 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:12.161850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:12.191865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9f3b9-79e4-40b7-9afc-c79e668b8f44 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:12.324887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:12.326440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:12.327663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:12.328127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] PATH_INFO: `/v3/projects/7471b564de914bd9bdc1208750075fe8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:12.333509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7471b564de914bd9bdc1208750075fe8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:12.336353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:12.419704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:12.438028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7471b564de914bd9bdc1208750075fe8', 'request_id': 'req-767c1e69-fe08-450d-a336-7428f4edcfb5'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:12.448325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7471b564de914bd9bdc1208750075fe8', 'request_id': 'req-767c1e69-fe08-450d-a336-7428f4edcfb5'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:12.459083 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7471b564de914bd9bdc1208750075fe8', 'request_id': 'req-767c1e69-fe08-450d-a336-7428f4edcfb5'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:12.488250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] The token cache is being invalidate because project 7471b564de914bd9bdc1208750075fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:12.488551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:12.489441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-767c1e69-fe08-450d-a336-7428f4edcfb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:12.493537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 908/1839] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:12 2022] DELETE /identity/v3/projects/7471b564de914bd9bdc1208750075fe8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:12.502885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b31f3-4a28-4a57-b872-580c0cda8e4c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:12.503765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b31f3-4a28-4a57-b872-580c0cda8e4c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:12.504049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b31f3-4a28-4a57-b872-580c0cda8e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:12.559757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b98b31f3-4a28-4a57-b872-580c0cda8e4c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:12.561234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b98b31f3-4a28-4a57-b872-580c0cda8e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:12.669761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 932/1840] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:12.678303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d2e2e-bd79-4881-a92e-c111563bc752 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:12.678595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d2e2e-bd79-4881-a92e-c111563bc752 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:12.679039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d2e2e-bd79-4881-a92e-c111563bc752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:12.707750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-889d2e2e-bd79-4881-a92e-c111563bc752 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:12.724849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-889d2e2e-bd79-4881-a92e-c111563bc752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:12.768135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b31f3-4a28-4a57-b872-580c0cda8e4c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:12.776574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 909/1841] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:00:12.782543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b31f3-4a28-4a57-b872-580c0cda8e4c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:12.792363 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85f55b-5746-4431-ab32-c8ca4d4b1966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:12.793366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85f55b-5746-4431-ab32-c8ca4d4b1966 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:12.793989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85f55b-5746-4431-ab32-c8ca4d4b1966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:12.794438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85f55b-5746-4431-ab32-c8ca4d4b1966 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:12.795722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85f55b-5746-4431-ab32-c8ca4d4b1966 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:00:12.797658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85f55b-5746-4431-ab32-c8ca4d4b1966 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:12.801016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85f55b-5746-4431-ab32-c8ca4d4b1966 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:12.883524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 933/1842] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:00:12.896957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d2e2e-bd79-4881-a92e-c111563bc752 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:12.922996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d2e2e-bd79-4881-a92e-c111563bc752 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:12.941967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6f0ce-1545-4c4a-9a19-ded9976355e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:12.944301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6f0ce-1545-4c4a-9a19-ded9976355e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:12.944828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6f0ce-1545-4c4a-9a19-ded9976355e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:12.945260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6f0ce-1545-4c4a-9a19-ded9976355e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:12.946190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6f0ce-1545-4c4a-9a19-ded9976355e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:12.948699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6f0ce-1545-4c4a-9a19-ded9976355e5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:13.002002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 910/1843] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:12 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:13.006711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85f55b-5746-4431-ab32-c8ca4d4b1966 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:13.031887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85f55b-5746-4431-ab32-c8ca4d4b1966 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:13.153806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1383de-601c-4ac1-95eb-852adfcb5863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:13.155293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1383de-601c-4ac1-95eb-852adfcb5863 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:13.155749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1383de-601c-4ac1-95eb-852adfcb5863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:13.156245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1383de-601c-4ac1-95eb-852adfcb5863 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:13.157316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1383de-601c-4ac1-95eb-852adfcb5863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:13.159666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1383de-601c-4ac1-95eb-852adfcb5863 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:13.190729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 934/1844] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:13 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:13.194986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6f0ce-1545-4c4a-9a19-ded9976355e5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:13.218473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6f0ce-1545-4c4a-9a19-ded9976355e5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:13.228161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3464a6-4329-476a-a956-28ce8cfe31ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:13.230649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3464a6-4329-476a-a956-28ce8cfe31ea admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:13.231075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3464a6-4329-476a-a956-28ce8cfe31ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:13.236900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3464a6-4329-476a-a956-28ce8cfe31ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:13.238005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3464a6-4329-476a-a956-28ce8cfe31ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:13.245138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3464a6-4329-476a-a956-28ce8cfe31ea admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:13.283684 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1383de-601c-4ac1-95eb-852adfcb5863 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:13.283829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 911/1845] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:13.307705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1383de-601c-4ac1-95eb-852adfcb5863 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:13.318593 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f121f2-54c0-4ac8-8bf5-d37dad69d70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:13.321905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f121f2-54c0-4ac8-8bf5-d37dad69d70e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:13.322271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f121f2-54c0-4ac8-8bf5-d37dad69d70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:13.322555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f121f2-54c0-4ac8-8bf5-d37dad69d70e admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:13.325222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f121f2-54c0-4ac8-8bf5-d37dad69d70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:13.330041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f121f2-54c0-4ac8-8bf5-d37dad69d70e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:13.355008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 935/1846] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:00:13.356290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3464a6-4329-476a-a956-28ce8cfe31ea admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:13.370066 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3464a6-4329-476a-a956-28ce8cfe31ea admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:13.385359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc7be-33c0-4d45-b223-60c9dfacf890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:13.389328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc7be-33c0-4d45-b223-60c9dfacf890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:13.389742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc7be-33c0-4d45-b223-60c9dfacf890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:13.390124 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc7be-33c0-4d45-b223-60c9dfacf890 admin admin] PATH_INFO: `/v3/projects/8e121b4ad34c463b97cd2f7a631719b8/users/629759a35b8f4a0391c39063b803ea5e/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:13.439651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdc7be-33c0-4d45-b223-60c9dfacf890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e121b4ad34c463b97cd2f7a631719b8, user_id=629759a35b8f4a0391c39063b803ea5e, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:13.444117 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdc7be-33c0-4d45-b223-60c9dfacf890 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:13.484220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 912/1847] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:13 2022] PUT /identity/v3/projects/8e121b4ad34c463b97cd2f7a631719b8/users/629759a35b8f4a0391c39063b803ea5e/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:13.496253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0bd66-7a80-4513-aee0-8d77e36f3f3d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:13.497348 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0bd66-7a80-4513-aee0-8d77e36f3f3d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:13.497638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0bd66-7a80-4513-aee0-8d77e36f3f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:13.567793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c7d0bd66-7a80-4513-aee0-8d77e36f3f3d None None] MFA Rules not processed for user `629759a35b8f4a0391c39063b803ea5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:13.569875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c7d0bd66-7a80-4513-aee0-8d77e36f3f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:13.586152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:13.694684 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 936/1848] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:13.699476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc7be-33c0-4d45-b223-60c9dfacf890 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:13.826828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc7be-33c0-4d45-b223-60c9dfacf890 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:13.844133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e894de-6e67-4409-b451-97f019209c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:13.855351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e894de-6e67-4409-b451-97f019209c7c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:13.855653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e894de-6e67-4409-b451-97f019209c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:13.855940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e894de-6e67-4409-b451-97f019209c7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:13.856656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e894de-6e67-4409-b451-97f019209c7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:13.858652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e894de-6e67-4409-b451-97f019209c7c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:13.923695 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 913/1849] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:13 2022] POST /identity/v3/projects => generated 373 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:13.928672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0bd66-7a80-4513-aee0-8d77e36f3f3d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:13.943341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0bd66-7a80-4513-aee0-8d77e36f3f3d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:14.086125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcf1a8-2517-4ac5-a2ff-079cb76615c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:14.087259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcf1a8-2517-4ac5-a2ff-079cb76615c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.088684 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcf1a8-2517-4ac5-a2ff-079cb76615c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.089051 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcf1a8-2517-4ac5-a2ff-079cb76615c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.093150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcf1a8-2517-4ac5-a2ff-079cb76615c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:14.100038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcf1a8-2517-4ac5-a2ff-079cb76615c8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:14.154577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 937/1850] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:13 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:14.156261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e894de-6e67-4409-b451-97f019209c7c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:14.180544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcf1a8-2517-4ac5-a2ff-079cb76615c8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:14.181147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e894de-6e67-4409-b451-97f019209c7c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:14.182417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-71bcf1a8-2517-4ac5-a2ff-079cb76615c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:14.193221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e36099-b6b9-4709-9236-4fb5406a5c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:14.197019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e36099-b6b9-4709-9236-4fb5406a5c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.197994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e36099-b6b9-4709-9236-4fb5406a5c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.202313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e36099-b6b9-4709-9236-4fb5406a5c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.204015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e36099-b6b9-4709-9236-4fb5406a5c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:14.206986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e36099-b6b9-4709-9236-4fb5406a5c11 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:14.208489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcf1a8-2517-4ac5-a2ff-079cb76615c8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:14.223002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 914/1851] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:14.226235 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3b6f3-c06d-4633-9489-e8d0ec1f25ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:14.226956 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b6f3-c06d-4633-9489-e8d0ec1f25ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.227623 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b6f3-c06d-4633-9489-e8d0ec1f25ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.228240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b6f3-c06d-4633-9489-e8d0ec1f25ed admin admin] PATH_INFO: `/v3/users/4ed870efee404214b9f4691f3fc0eb3a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.228753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e36099-b6b9-4709-9236-4fb5406a5c11 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:14.229213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3b6f3-c06d-4633-9489-e8d0ec1f25ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed870efee404214b9f4691f3fc0eb3a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:14.231358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3b6f3-c06d-4633-9489-e8d0ec1f25ed admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:14.242668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e36099-b6b9-4709-9236-4fb5406a5c11 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:14.249726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574bf5f-41b2-46c8-ac6a-d1550e9eda5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:14.252395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574bf5f-41b2-46c8-ac6a-d1550e9eda5b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.252945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574bf5f-41b2-46c8-ac6a-d1550e9eda5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.253550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574bf5f-41b2-46c8-ac6a-d1550e9eda5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.254487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574bf5f-41b2-46c8-ac6a-d1550e9eda5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:14.256372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574bf5f-41b2-46c8-ac6a-d1550e9eda5b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:14.271068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 915/1852] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:14.275273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574bf5f-41b2-46c8-ac6a-d1550e9eda5b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:14.291895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574bf5f-41b2-46c8-ac6a-d1550e9eda5b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:14.300675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e3169-6bbb-4092-bd6c-343b747ceac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:14.301979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e3169-6bbb-4092-bd6c-343b747ceac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.302496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e3169-6bbb-4092-bd6c-343b747ceac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.303221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e3169-6bbb-4092-bd6c-343b747ceac8 admin admin] PATH_INFO: `/v3/projects/86986f85e3224f8a8ffb52a300a89a63/users/73924a1092d14885939cf0137b012437/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.328297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:14.333116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fea3b6f3-c06d-4633-9489-e8d0ec1f25ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed870efee404214b9f4691f3fc0eb3a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:14.333873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fea3b6f3-c06d-4633-9489-e8d0ec1f25ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed870efee404214b9f4691f3fc0eb3a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:14.338413 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e3169-6bbb-4092-bd6c-343b747ceac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86986f85e3224f8a8ffb52a300a89a63, user_id=73924a1092d14885939cf0137b012437, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:14.341716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e3169-6bbb-4092-bd6c-343b747ceac8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:14.371444 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 916/1853] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:14 2022] PUT /identity/v3/projects/86986f85e3224f8a8ffb52a300a89a63/users/73924a1092d14885939cf0137b012437/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:14.376866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86390677-60da-482b-894b-37c0e37cc221 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.378309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fea3b6f3-c06d-4633-9489-e8d0ec1f25ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed870efee404214b9f4691f3fc0eb3a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:14.381128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86390677-60da-482b-894b-37c0e37cc221 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.381375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86390677-60da-482b-894b-37c0e37cc221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.395963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fea3b6f3-c06d-4633-9489-e8d0ec1f25ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed870efee404214b9f4691f3fc0eb3a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:14.408706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fea3b6f3-c06d-4633-9489-e8d0ec1f25ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed870efee404214b9f4691f3fc0eb3a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:14.425540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 938/1854] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:14 2022] DELETE /identity/v3/users/4ed870efee404214b9f4691f3fc0eb3a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:14.431505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e986f3-42c8-424f-bc7e-b09a3024d9e1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.432097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e986f3-42c8-424f-bc7e-b09a3024d9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.432534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e986f3-42c8-424f-bc7e-b09a3024d9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.450883 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-86390677-60da-482b-894b-37c0e37cc221 None None] MFA Rules not processed for user `73924a1092d14885939cf0137b012437`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:14.452244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-86390677-60da-482b-894b-37c0e37cc221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:14.460450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c6e986f3-42c8-424f-bc7e-b09a3024d9e1 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:14.462367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c6e986f3-42c8-424f-bc7e-b09a3024d9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:14.476497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:14.568932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 939/1855] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:14.607109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 917/1856] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:14.608080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e986f3-42c8-424f-bc7e-b09a3024d9e1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:14.610170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c6e986f3-42c8-424f-bc7e-b09a3024d9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:14.625006 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e986f3-42c8-424f-bc7e-b09a3024d9e1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:14.630524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb6b57-0d0e-47a9-a832-af0530dbb98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:14.632050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb6b57-0d0e-47a9-a832-af0530dbb98c admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.632644 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb6b57-0d0e-47a9-a832-af0530dbb98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.633124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb6b57-0d0e-47a9-a832-af0530dbb98c admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.634264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86390677-60da-482b-894b-37c0e37cc221 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:14.635919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-86390677-60da-482b-894b-37c0e37cc221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:14.645925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb6b57-0d0e-47a9-a832-af0530dbb98c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:14.651003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb6b57-0d0e-47a9-a832-af0530dbb98c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:14.720975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 940/1857] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:14 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:00:14.722327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb6b57-0d0e-47a9-a832-af0530dbb98c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:14.744142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86390677-60da-482b-894b-37c0e37cc221 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:14.806774 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afe05c-97b6-4c31-8acf-fdf4b5b058c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:14.807453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe05c-97b6-4c31-8acf-fdf4b5b058c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.807728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe05c-97b6-4c31-8acf-fdf4b5b058c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.808539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe05c-97b6-4c31-8acf-fdf4b5b058c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.826374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb6b57-0d0e-47a9-a832-af0530dbb98c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:14.831941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d78a1a-eec2-4855-9a1e-d59981a3f2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:14.832652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d78a1a-eec2-4855-9a1e-d59981a3f2dc admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.833002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d78a1a-eec2-4855-9a1e-d59981a3f2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.833347 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d78a1a-eec2-4855-9a1e-d59981a3f2dc admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.834278 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d78a1a-eec2-4855-9a1e-d59981a3f2dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:14.836754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d78a1a-eec2-4855-9a1e-d59981a3f2dc admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:14.869947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 941/1858] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:14 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:14.872493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d78a1a-eec2-4855-9a1e-d59981a3f2dc admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:14.885671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d78a1a-eec2-4855-9a1e-d59981a3f2dc admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:14.893091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd37cd-f0c6-421c-9f8c-2299f9c23e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:14.894061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd37cd-f0c6-421c-9f8c-2299f9c23e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.894653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd37cd-f0c6-421c-9f8c-2299f9c23e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.895087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd37cd-f0c6-421c-9f8c-2299f9c23e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.896054 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd37cd-f0c6-421c-9f8c-2299f9c23e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:14.898326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd37cd-f0c6-421c-9f8c-2299f9c23e0f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:14.915773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:14.915910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 942/1859] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:14.917213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd37cd-f0c6-421c-9f8c-2299f9c23e0f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:14.934073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd37cd-f0c6-421c-9f8c-2299f9c23e0f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:14.935261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afe05c-97b6-4c31-8acf-fdf4b5b058c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:14.936585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afe05c-97b6-4c31-8acf-fdf4b5b058c8 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:14.939524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d34e1a-92c5-4124-941f-128a16b00f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:14.940481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d34e1a-92c5-4124-941f-128a16b00f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:14.940772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d34e1a-92c5-4124-941f-128a16b00f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:14.941301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d34e1a-92c5-4124-941f-128a16b00f27 admin admin] PATH_INFO: `/v3/projects/8e3789199cfc425b85691753b38f375a/users/e1203d3ff9f047e9b4f44572bcff3698/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:14.950726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 918/1860] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:14.960899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d34e1a-92c5-4124-941f-128a16b00f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e3789199cfc425b85691753b38f375a, user_id=e1203d3ff9f047e9b4f44572bcff3698, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:14.963853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d34e1a-92c5-4124-941f-128a16b00f27 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:14.987285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 943/1861] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:14 2022] PUT /identity/v3/projects/8e3789199cfc425b85691753b38f375a/users/e1203d3ff9f047e9b4f44572bcff3698/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:14.991556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afe05c-97b6-4c31-8acf-fdf4b5b058c8 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:15.008198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afe05c-97b6-4c31-8acf-fdf4b5b058c8 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:15.228765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a846d-aa43-4ac1-95df-869a827dcf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:15.230259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a846d-aa43-4ac1-95df-869a827dcf96 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:15.230752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a846d-aa43-4ac1-95df-869a827dcf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:15.231198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a846d-aa43-4ac1-95df-869a827dcf96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:15.232051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a846d-aa43-4ac1-95df-869a827dcf96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:15.234395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a846d-aa43-4ac1-95df-869a827dcf96 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:15.249860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 919/1862] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:15.266506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d34e1a-92c5-4124-941f-128a16b00f27 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:15.271438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499047f-8be0-468a-9d24-5a93fd1ebe86 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:15.271438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499047f-8be0-468a-9d24-5a93fd1ebe86 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:15.271902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499047f-8be0-468a-9d24-5a93fd1ebe86 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:15.273501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 920/1863] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jul 28 04:00:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 04:00:15.275827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d34e1a-92c5-4124-941f-128a16b00f27 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:15.282947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41690aef-7eb0-41d4-948f-0cdc5f5bedb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:15.283930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41690aef-7eb0-41d4-948f-0cdc5f5bedb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:15.284387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41690aef-7eb0-41d4-948f-0cdc5f5bedb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:15.285023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41690aef-7eb0-41d4-948f-0cdc5f5bedb8 admin admin] PATH_INFO: `/v3/projects/8e3789199cfc425b85691753b38f375a/users/e1203d3ff9f047e9b4f44572bcff3698/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:15.287457 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41690aef-7eb0-41d4-948f-0cdc5f5bedb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e3789199cfc425b85691753b38f375a, user_id=e1203d3ff9f047e9b4f44572bcff3698, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:15.290951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41690aef-7eb0-41d4-948f-0cdc5f5bedb8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:15.320484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df9eb0-4102-4990-b639-621428fbcbc9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:15.320869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df9eb0-4102-4990-b639-621428fbcbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:15.321164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df9eb0-4102-4990-b639-621428fbcbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:15.325214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 944/1864] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:15 2022] PUT /identity/v3/projects/8e3789199cfc425b85691753b38f375a/users/e1203d3ff9f047e9b4f44572bcff3698/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:15.333925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe777c-3a7f-4313-964c-bd8f2f70650f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:15.335078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe777c-3a7f-4313-964c-bd8f2f70650f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:15.335078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe777c-3a7f-4313-964c-bd8f2f70650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:15.379418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-81fe777c-3a7f-4313-964c-bd8f2f70650f None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:15.381954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-81fe777c-3a7f-4313-964c-bd8f2f70650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:15.384194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-90df9eb0-4102-4990-b639-621428fbcbc9 None None] MFA Rules not processed for user `e1203d3ff9f047e9b4f44572bcff3698`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:15.386139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-90df9eb0-4102-4990-b639-621428fbcbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:15.475527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:15.521643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 921/1865] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:15.534742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 945/1866] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jul 28 04:00:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:15.535969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df9eb0-4102-4990-b639-621428fbcbc9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:15.537381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-90df9eb0-4102-4990-b639-621428fbcbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:15.619130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df9eb0-4102-4990-b639-621428fbcbc9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:15.624234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33280d2-dafc-4e6c-be91-6f93782ed96b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:15.624935 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33280d2-dafc-4e6c-be91-6f93782ed96b service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:15.625308 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33280d2-dafc-4e6c-be91-6f93782ed96b service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:15.625849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33280d2-dafc-4e6c-be91-6f93782ed96b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:15.629512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c33280d2-dafc-4e6c-be91-6f93782ed96b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:15.642050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe777c-3a7f-4313-964c-bd8f2f70650f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:15.644143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-81fe777c-3a7f-4313-964c-bd8f2f70650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:15.671278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:15.683011 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe777c-3a7f-4313-964c-bd8f2f70650f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:15.702219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b399c-c95c-4ee1-84fd-b0563cdb859c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:15.703059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b399c-c95c-4ee1-84fd-b0563cdb859c service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:15.703540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b399c-c95c-4ee1-84fd-b0563cdb859c service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:15.704020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b399c-c95c-4ee1-84fd-b0563cdb859c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:15.725838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b399c-c95c-4ee1-84fd-b0563cdb859c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:15.729745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b399c-c95c-4ee1-84fd-b0563cdb859c service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:15.739197 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33280d2-dafc-4e6c-be91-6f93782ed96b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:15.741476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33280d2-dafc-4e6c-be91-6f93782ed96b service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:15.749738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 946/1867] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:00:15.752910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 922/1868] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:16.005502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b399c-c95c-4ee1-84fd-b0563cdb859c service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:16.007482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1b5b399c-c95c-4ee1-84fd-b0563cdb859c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:16.106155 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b399c-c95c-4ee1-84fd-b0563cdb859c service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:16.111799 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:16.112447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:16.112735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:16.113014 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] PATH_INFO: `/v3/projects/7c005adc61db4faf8803b57f819e2461` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:16.114193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c005adc61db4faf8803b57f819e2461)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:16.116171 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:16.164737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c005adc61db4faf8803b57f819e2461', 'request_id': 'req-fe60cb72-d399-4667-96b7-30779c82bb94'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.170215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c005adc61db4faf8803b57f819e2461', 'request_id': 'req-fe60cb72-d399-4667-96b7-30779c82bb94'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.176654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c005adc61db4faf8803b57f819e2461', 'request_id': 'req-fe60cb72-d399-4667-96b7-30779c82bb94'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.188542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] The token cache is being invalidate because project 7c005adc61db4faf8803b57f819e2461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:16.188833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.189467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.191250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 947/1869] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:16 2022] DELETE /identity/v3/projects/7c005adc61db4faf8803b57f819e2461 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:16.195077 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33280d2-dafc-4e6c-be91-6f93782ed96b service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:16.206568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33280d2-dafc-4e6c-be91-6f93782ed96b service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:16.264049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac496e1e-5971-48c8-8e31-180a6f38a94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:16.264712 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496e1e-5971-48c8-8e31-180a6f38a94a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:16.264958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496e1e-5971-48c8-8e31-180a6f38a94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:16.265231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496e1e-5971-48c8-8e31-180a6f38a94a admin admin] PATH_INFO: `/v3/users/8e58adb7ee704d53ba8a080927771c08` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:16.266461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac496e1e-5971-48c8-8e31-180a6f38a94a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e58adb7ee704d53ba8a080927771c08)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:16.268453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac496e1e-5971-48c8-8e31-180a6f38a94a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:16.317055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ac496e1e-5971-48c8-8e31-180a6f38a94a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e58adb7ee704d53ba8a080927771c08'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.317709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ac496e1e-5971-48c8-8e31-180a6f38a94a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e58adb7ee704d53ba8a080927771c08'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.324985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ac496e1e-5971-48c8-8e31-180a6f38a94a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e58adb7ee704d53ba8a080927771c08'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.337401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ac496e1e-5971-48c8-8e31-180a6f38a94a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e58adb7ee704d53ba8a080927771c08'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.344036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ac496e1e-5971-48c8-8e31-180a6f38a94a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e58adb7ee704d53ba8a080927771c08'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.349861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 923/1870] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:16 2022] DELETE /identity/v3/users/8e58adb7ee704d53ba8a080927771c08 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:16.628411 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:16.629917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:16.646750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60cb72-d399-4667-96b7-30779c82bb94 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:16.706867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:16.731336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:16.732241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:16.732796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:16.733268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] PATH_INFO: `/v3/projects/8197e5897bdf489f936cfe5e07456fec` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:16.736647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8197e5897bdf489f936cfe5e07456fec)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:16.738132 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:16.814059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8197e5897bdf489f936cfe5e07456fec', 'request_id': 'req-14cdc5f7-5191-4104-b842-6a4cdc66e206'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.819973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8197e5897bdf489f936cfe5e07456fec', 'request_id': 'req-14cdc5f7-5191-4104-b842-6a4cdc66e206'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.826575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8197e5897bdf489f936cfe5e07456fec', 'request_id': 'req-14cdc5f7-5191-4104-b842-6a4cdc66e206'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.848626 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] The token cache is being invalidate because project 8197e5897bdf489f936cfe5e07456fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:16.848951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.849659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-14cdc5f7-5191-4104-b842-6a4cdc66e206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:16.851246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 948/1871] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:16 2022] DELETE /identity/v3/projects/8197e5897bdf489f936cfe5e07456fec => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:16.856024 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beeb3b5-1c0e-4413-a5e9-dabcb47f17fc None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:16.856310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beeb3b5-1c0e-4413-a5e9-dabcb47f17fc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:16.856620 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beeb3b5-1c0e-4413-a5e9-dabcb47f17fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:16.876996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3beeb3b5-1c0e-4413-a5e9-dabcb47f17fc None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:16.878377 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3beeb3b5-1c0e-4413-a5e9-dabcb47f17fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:16.918154 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:16.945713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 924/1872] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:16.954055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a06ea6-4268-45bc-89f6-7b69b11c22ec None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:16.954351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a06ea6-4268-45bc-89f6-7b69b11c22ec None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:16.954630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a06ea6-4268-45bc-89f6-7b69b11c22ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:16.974947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-29a06ea6-4268-45bc-89f6-7b69b11c22ec None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:16.976164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-29a06ea6-4268-45bc-89f6-7b69b11c22ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:16.984281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 949/1873] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:16.988551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beeb3b5-1c0e-4413-a5e9-dabcb47f17fc None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:16.998321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beeb3b5-1c0e-4413-a5e9-dabcb47f17fc None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.003821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b66fe-12e3-4884-967a-8e16cc15b3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.004446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b66fe-12e3-4884-967a-8e16cc15b3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.004735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b66fe-12e3-4884-967a-8e16cc15b3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.005013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b66fe-12e3-4884-967a-8e16cc15b3e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.005721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b66fe-12e3-4884-967a-8e16cc15b3e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:00:17.006147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b66fe-12e3-4884-967a-8e16cc15b3e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.007918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b66fe-12e3-4884-967a-8e16cc15b3e3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.020469 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 925/1874] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:00:17.024846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a06ea6-4268-45bc-89f6-7b69b11c22ec None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:17.034587 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a06ea6-4268-45bc-89f6-7b69b11c22ec None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.039481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68387a8-7248-4b2f-9737-63ab2018d2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.040115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68387a8-7248-4b2f-9737-63ab2018d2b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.040420 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68387a8-7248-4b2f-9737-63ab2018d2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.040699 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68387a8-7248-4b2f-9737-63ab2018d2b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.041374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68387a8-7248-4b2f-9737-63ab2018d2b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.043345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68387a8-7248-4b2f-9737-63ab2018d2b3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.075475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 950/1875] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:17 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:00:17.076667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b66fe-12e3-4884-967a-8e16cc15b3e3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:17.087892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b66fe-12e3-4884-967a-8e16cc15b3e3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.156724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9823a9-a97c-4cc2-974b-f89e301b4439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.157335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9823a9-a97c-4cc2-974b-f89e301b4439 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.157618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9823a9-a97c-4cc2-974b-f89e301b4439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.157889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9823a9-a97c-4cc2-974b-f89e301b4439 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.158566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9823a9-a97c-4cc2-974b-f89e301b4439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.160503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9823a9-a97c-4cc2-974b-f89e301b4439 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.181969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 926/1876] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:17 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:00:17.186109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68387a8-7248-4b2f-9737-63ab2018d2b3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:17.196969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68387a8-7248-4b2f-9737-63ab2018d2b3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.201532 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865964-aba6-47af-bcd4-e77cf1db5355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.202139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865964-aba6-47af-bcd4-e77cf1db5355 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.202420 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865964-aba6-47af-bcd4-e77cf1db5355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.202694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865964-aba6-47af-bcd4-e77cf1db5355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.204027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28865964-aba6-47af-bcd4-e77cf1db5355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.205618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28865964-aba6-47af-bcd4-e77cf1db5355 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.215364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 951/1877] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:17.219357 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9823a9-a97c-4cc2-974b-f89e301b4439 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:17.229060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9823a9-a97c-4cc2-974b-f89e301b4439 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.233634 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0c93c-9b89-45ac-84a5-58c10148653e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.234234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0c93c-9b89-45ac-84a5-58c10148653e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.234565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0c93c-9b89-45ac-84a5-58c10148653e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.234878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0c93c-9b89-45ac-84a5-58c10148653e admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.235583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0c93c-9b89-45ac-84a5-58c10148653e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.237163 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0c93c-9b89-45ac-84a5-58c10148653e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.247601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 927/1878] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:17.252223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865964-aba6-47af-bcd4-e77cf1db5355 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:17.262124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865964-aba6-47af-bcd4-e77cf1db5355 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.267414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d93633-ebf0-46bb-950d-05b8903f6a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.268068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d93633-ebf0-46bb-950d-05b8903f6a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.268355 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d93633-ebf0-46bb-950d-05b8903f6a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.268640 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d93633-ebf0-46bb-950d-05b8903f6a6b admin admin] PATH_INFO: `/v3/projects/f78ad65de49648d98a22b3e0eb083a35/users/3a3e207f301940259ee3eaeb190d422b/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.284769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d93633-ebf0-46bb-950d-05b8903f6a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f78ad65de49648d98a22b3e0eb083a35, user_id=3a3e207f301940259ee3eaeb190d422b, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.287617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d93633-ebf0-46bb-950d-05b8903f6a6b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.300722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 952/1879] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:17 2022] PUT /identity/v3/projects/f78ad65de49648d98a22b3e0eb083a35/users/3a3e207f301940259ee3eaeb190d422b/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:17.310920 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f32388-19a9-485d-b1d0-497b51cd06ab None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.311417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f32388-19a9-485d-b1d0-497b51cd06ab None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.311534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f32388-19a9-485d-b1d0-497b51cd06ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.340793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-41f32388-19a9-485d-b1d0-497b51cd06ab None None] MFA Rules not processed for user `3a3e207f301940259ee3eaeb190d422b`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:17.342016 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-41f32388-19a9-485d-b1d0-497b51cd06ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:17.409518 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 928/1880] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:17.414244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d93633-ebf0-46bb-950d-05b8903f6a6b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:17.478483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d93633-ebf0-46bb-950d-05b8903f6a6b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.483217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ab802-fa2e-48db-a656-c9e06195b5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.483838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab802-fa2e-48db-a656-c9e06195b5ca admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.484123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab802-fa2e-48db-a656-c9e06195b5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.484402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab802-fa2e-48db-a656-c9e06195b5ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.485745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ab802-fa2e-48db-a656-c9e06195b5ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.487698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ab802-fa2e-48db-a656-c9e06195b5ca admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.502641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:17.515330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 953/1881] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:17 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:00:17.519964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f32388-19a9-485d-b1d0-497b51cd06ab None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:17.530803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f32388-19a9-485d-b1d0-497b51cd06ab None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.564298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:17.588726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80063614-e863-4997-a1dc-3b9e0f5c2e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.589346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80063614-e863-4997-a1dc-3b9e0f5c2e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.589628 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80063614-e863-4997-a1dc-3b9e0f5c2e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.589926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80063614-e863-4997-a1dc-3b9e0f5c2e88 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.590591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80063614-e863-4997-a1dc-3b9e0f5c2e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.592594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80063614-e863-4997-a1dc-3b9e0f5c2e88 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.613378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 929/1882] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:17 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:17.618577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ab802-fa2e-48db-a656-c9e06195b5ca admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:17.631251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ab802-fa2e-48db-a656-c9e06195b5ca admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.636925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77237eab-cfd0-4f91-b7cc-ee08f3537c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.637806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77237eab-cfd0-4f91-b7cc-ee08f3537c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.638251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77237eab-cfd0-4f91-b7cc-ee08f3537c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.638526 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77237eab-cfd0-4f91-b7cc-ee08f3537c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.639265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77237eab-cfd0-4f91-b7cc-ee08f3537c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.640957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77237eab-cfd0-4f91-b7cc-ee08f3537c3f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.653098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 954/1883] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:17.657428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80063614-e863-4997-a1dc-3b9e0f5c2e88 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:17.667641 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80063614-e863-4997-a1dc-3b9e0f5c2e88 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.672902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1bcca-c871-4909-853f-4b75496d8b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.673563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1bcca-c871-4909-853f-4b75496d8b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.673848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1bcca-c871-4909-853f-4b75496d8b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.674131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1bcca-c871-4909-853f-4b75496d8b1f admin admin] PATH_INFO: `/v3/projects/97db4c5972fd4285b8fee56651504f04/users/5de06169fbc645718d46abcccc374b8b/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.689454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1bcca-c871-4909-853f-4b75496d8b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97db4c5972fd4285b8fee56651504f04, user_id=5de06169fbc645718d46abcccc374b8b, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.692143 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1bcca-c871-4909-853f-4b75496d8b1f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.704680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 930/1884] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:17 2022] PUT /identity/v3/projects/97db4c5972fd4285b8fee56651504f04/users/5de06169fbc645718d46abcccc374b8b/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:17.708700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77237eab-cfd0-4f91-b7cc-ee08f3537c3f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:17.718549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77237eab-cfd0-4f91-b7cc-ee08f3537c3f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.777472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039341f-a112-4312-8598-1272ddd41b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.778500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039341f-a112-4312-8598-1272ddd41b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.778500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039341f-a112-4312-8598-1272ddd41b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.778500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039341f-a112-4312-8598-1272ddd41b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.779259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039341f-a112-4312-8598-1272ddd41b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.781043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039341f-a112-4312-8598-1272ddd41b83 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.792543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 955/1885] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:17.798748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1bcca-c871-4909-853f-4b75496d8b1f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:17.810824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1bcca-c871-4909-853f-4b75496d8b1f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:17.816706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb2a0-df81-4421-94e4-0c26156b48c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:17.818205 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb2a0-df81-4421-94e4-0c26156b48c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.818560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb2a0-df81-4421-94e4-0c26156b48c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.818951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb2a0-df81-4421-94e4-0c26156b48c1 admin admin] PATH_INFO: `/v3/projects/97db4c5972fd4285b8fee56651504f04/users/5de06169fbc645718d46abcccc374b8b/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.821168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacb2a0-df81-4421-94e4-0c26156b48c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97db4c5972fd4285b8fee56651504f04, user_id=5de06169fbc645718d46abcccc374b8b, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:17.823988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacb2a0-df81-4421-94e4-0c26156b48c1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:17.835393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 931/1886] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:17 2022] PUT /identity/v3/projects/97db4c5972fd4285b8fee56651504f04/users/5de06169fbc645718d46abcccc374b8b/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:17.847709 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67f2c2-2c65-490d-96be-ff126237b3f9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:17.848004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67f2c2-2c65-490d-96be-ff126237b3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:17.848282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67f2c2-2c65-490d-96be-ff126237b3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:17.880896 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8d67f2c2-2c65-490d-96be-ff126237b3f9 None None] MFA Rules not processed for user `5de06169fbc645718d46abcccc374b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:17.882291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8d67f2c2-2c65-490d-96be-ff126237b3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:17.997677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 956/1887] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:17 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:18.011574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb2a0-df81-4421-94e4-0c26156b48c1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:18.014555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5aacb2a0-df81-4421-94e4-0c26156b48c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:18.087165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb2a0-df81-4421-94e4-0c26156b48c1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:18.091244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34473ca9-9cae-4278-9ff2-e61aa11ba111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:18.091875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34473ca9-9cae-4278-9ff2-e61aa11ba111 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:18.092161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34473ca9-9cae-4278-9ff2-e61aa11ba111 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:18.092436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34473ca9-9cae-4278-9ff2-e61aa11ba111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:18.102865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34473ca9-9cae-4278-9ff2-e61aa11ba111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:18.104313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34473ca9-9cae-4278-9ff2-e61aa11ba111 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:18.114954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 932/1888] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:18 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:18.208816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67f2c2-2c65-490d-96be-ff126237b3f9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:18.218680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67f2c2-2c65-490d-96be-ff126237b3f9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:18.222944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd33e0-3cc9-4e89-828c-7192fb2a43dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:18.223627 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd33e0-3cc9-4e89-828c-7192fb2a43dd service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:18.224081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd33e0-3cc9-4e89-828c-7192fb2a43dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:18.224483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd33e0-3cc9-4e89-828c-7192fb2a43dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:18.286970 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd33e0-3cc9-4e89-828c-7192fb2a43dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:18.288686 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd33e0-3cc9-4e89-828c-7192fb2a43dd service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:18.298539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 957/1889] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:19.448206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34473ca9-9cae-4278-9ff2-e61aa11ba111 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:19.456437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34473ca9-9cae-4278-9ff2-e61aa11ba111 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:19.460029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5648a51-cebb-4d05-affd-8dad3081fe0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:19.460633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5648a51-cebb-4d05-affd-8dad3081fe0c service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:19.460913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5648a51-cebb-4d05-affd-8dad3081fe0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:19.461186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5648a51-cebb-4d05-affd-8dad3081fe0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:19.462931 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e5648a51-cebb-4d05-affd-8dad3081fe0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:19.521809 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5648a51-cebb-4d05-affd-8dad3081fe0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:19.523280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5648a51-cebb-4d05-affd-8dad3081fe0c service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:19.532700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 933/1890] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:20.237079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd33e0-3cc9-4e89-828c-7192fb2a43dd service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:20.239277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-62bd33e0-3cc9-4e89-828c-7192fb2a43dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:20.248553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:20.255208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd33e0-3cc9-4e89-828c-7192fb2a43dd service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:20.260741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc890c-61d7-4bea-bb00-fe06b74bbe28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:20.261926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc890c-61d7-4bea-bb00-fe06b74bbe28 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:20.263788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc890c-61d7-4bea-bb00-fe06b74bbe28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:20.263788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc890c-61d7-4bea-bb00-fe06b74bbe28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:20.265675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-71fc890c-61d7-4bea-bb00-fe06b74bbe28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:20.373603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc890c-61d7-4bea-bb00-fe06b74bbe28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:20.375812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc890c-61d7-4bea-bb00-fe06b74bbe28 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:20.388482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 958/1891] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:24.130927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5648a51-cebb-4d05-affd-8dad3081fe0c service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:24.132947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e5648a51-cebb-4d05-affd-8dad3081fe0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:24.178445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:24.199219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc890c-61d7-4bea-bb00-fe06b74bbe28 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:24.223475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5648a51-cebb-4d05-affd-8dad3081fe0c service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:24.228659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542cf33-9d33-40a9-813e-5af3b9ec669e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:24.229420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542cf33-9d33-40a9-813e-5af3b9ec669e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:24.229775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542cf33-9d33-40a9-813e-5af3b9ec669e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:24.230125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542cf33-9d33-40a9-813e-5af3b9ec669e admin admin] PATH_INFO: `/v3/users/005ddb1d20df40d9897d4fa625fdb9f2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:24.231442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542cf33-9d33-40a9-813e-5af3b9ec669e admin admin] RBAC: Authorizing `identity:delete_user(user_id=005ddb1d20df40d9897d4fa625fdb9f2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:24.233462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542cf33-9d33-40a9-813e-5af3b9ec669e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:24.278208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc890c-61d7-4bea-bb00-fe06b74bbe28 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:24.284117 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830abf3-1bad-45e7-a4b9-badc5f076f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:24.285104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830abf3-1bad-45e7-a4b9-badc5f076f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:24.285541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830abf3-1bad-45e7-a4b9-badc5f076f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:24.286212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830abf3-1bad-45e7-a4b9-badc5f076f46 admin admin] PATH_INFO: `/v3/users/3a3e207f301940259ee3eaeb190d422b` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:24.287706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830abf3-1bad-45e7-a4b9-badc5f076f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3e207f301940259ee3eaeb190d422b)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:24.289112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f542cf33-9d33-40a9-813e-5af3b9ec669e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '005ddb1d20df40d9897d4fa625fdb9f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.289840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f542cf33-9d33-40a9-813e-5af3b9ec669e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '005ddb1d20df40d9897d4fa625fdb9f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.290604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830abf3-1bad-45e7-a4b9-badc5f076f46 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:24.300972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f542cf33-9d33-40a9-813e-5af3b9ec669e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '005ddb1d20df40d9897d4fa625fdb9f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.325846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f542cf33-9d33-40a9-813e-5af3b9ec669e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '005ddb1d20df40d9897d4fa625fdb9f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.335367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f542cf33-9d33-40a9-813e-5af3b9ec669e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '005ddb1d20df40d9897d4fa625fdb9f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.343605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 934/1892] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:24 2022] DELETE /identity/v3/users/005ddb1d20df40d9897d4fa625fdb9f2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:24.348883 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bac0d6-f770-4cba-9292-4df2a96ed1f7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:24.349400 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bac0d6-f770-4cba-9292-4df2a96ed1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:24.349755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bac0d6-f770-4cba-9292-4df2a96ed1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:24.355768 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d830abf3-1bad-45e7-a4b9-badc5f076f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3e207f301940259ee3eaeb190d422b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.356434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d830abf3-1bad-45e7-a4b9-badc5f076f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3e207f301940259ee3eaeb190d422b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.369584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-68bac0d6-f770-4cba-9292-4df2a96ed1f7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:24.370972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-68bac0d6-f770-4cba-9292-4df2a96ed1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:24.372732 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d830abf3-1bad-45e7-a4b9-badc5f076f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3e207f301940259ee3eaeb190d422b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.384036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d830abf3-1bad-45e7-a4b9-badc5f076f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3e207f301940259ee3eaeb190d422b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.394491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d830abf3-1bad-45e7-a4b9-badc5f076f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3e207f301940259ee3eaeb190d422b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.404158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 959/1893] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:24 2022] DELETE /identity/v3/users/3a3e207f301940259ee3eaeb190d422b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:24.409875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae90ec2-6220-4050-91e0-24de7739e2b9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:24.410247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae90ec2-6220-4050-91e0-24de7739e2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:24.410604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae90ec2-6220-4050-91e0-24de7739e2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:24.444302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1ae90ec2-6220-4050-91e0-24de7739e2b9 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:24.446507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1ae90ec2-6220-4050-91e0-24de7739e2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:24.464951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:24.474204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 935/1894] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:24.487219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bac0d6-f770-4cba-9292-4df2a96ed1f7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:24.490210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-68bac0d6-f770-4cba-9292-4df2a96ed1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:24.541192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 960/1895] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:00:24.557845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae90ec2-6220-4050-91e0-24de7739e2b9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:24.559316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1ae90ec2-6220-4050-91e0-24de7739e2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:24.594838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bac0d6-f770-4cba-9292-4df2a96ed1f7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:24.599807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:24.665380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f394c59-e9a0-42ce-ab5e-7a2e92f1d494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:24.666180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f394c59-e9a0-42ce-ab5e-7a2e92f1d494 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:24.666605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f394c59-e9a0-42ce-ab5e-7a2e92f1d494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:24.667224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f394c59-e9a0-42ce-ab5e-7a2e92f1d494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:24.680731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f394c59-e9a0-42ce-ab5e-7a2e92f1d494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:24.682804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f394c59-e9a0-42ce-ab5e-7a2e92f1d494 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:24.694249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae90ec2-6220-4050-91e0-24de7739e2b9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:24.699756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 936/1896] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:24.704544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6a6db-8974-40db-84c5-d787b9c9edb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:24.707624 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6a6db-8974-40db-84c5-d787b9c9edb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:24.707926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6a6db-8974-40db-84c5-d787b9c9edb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:24.708211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6a6db-8974-40db-84c5-d787b9c9edb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:24.724392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6a6db-8974-40db-84c5-d787b9c9edb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:24.726274 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6a6db-8974-40db-84c5-d787b9c9edb9 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:24.741770 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 961/1897] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:00:24.831701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f394c59-e9a0-42ce-ab5e-7a2e92f1d494 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:24.833261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7f394c59-e9a0-42ce-ab5e-7a2e92f1d494 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:24.847322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f394c59-e9a0-42ce-ab5e-7a2e92f1d494 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:24.853593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:24.854391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:24.854764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:24.855138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] PATH_INFO: `/v3/users/629759a35b8f4a0391c39063b803ea5e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:24.856468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=629759a35b8f4a0391c39063b803ea5e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:24.858580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:24.923101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '629759a35b8f4a0391c39063b803ea5e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.923837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '629759a35b8f4a0391c39063b803ea5e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.938889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '629759a35b8f4a0391c39063b803ea5e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.955951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '629759a35b8f4a0391c39063b803ea5e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.964114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '629759a35b8f4a0391c39063b803ea5e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:24.972678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 937/1898] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:24 2022] DELETE /identity/v3/users/629759a35b8f4a0391c39063b803ea5e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:24.977240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db860df-081a-4f5f-a493-801b558ee312 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:24.978432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db860df-081a-4f5f-a493-801b558ee312 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:24.978432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db860df-081a-4f5f-a493-801b558ee312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:25.009798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1db860df-081a-4f5f-a493-801b558ee312 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:25.011241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1db860df-081a-4f5f-a493-801b558ee312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:25.111934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 962/1899] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:25.119922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:25.121586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:25.189437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e998b4-b3ae-40c3-bbd7-2cddad689d8d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:25.194330 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e472c-2aa4-4be6-b087-576034e931a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:25.195000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e472c-2aa4-4be6-b087-576034e931a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:25.195286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e472c-2aa4-4be6-b087-576034e931a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:25.195563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e472c-2aa4-4be6-b087-576034e931a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:25.210286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e472c-2aa4-4be6-b087-576034e931a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:25.211945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e472c-2aa4-4be6-b087-576034e931a1 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:25.246540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 938/1900] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:00:25.472552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72aa85-a27b-43d6-9da2-b70c49efadd5 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:25.473002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72aa85-a27b-43d6-9da2-b70c49efadd5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:25.473116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72aa85-a27b-43d6-9da2-b70c49efadd5 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:25.474528 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 963/1901] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jul 28 04:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 04:00:25.524388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fd42a-7a08-4e3d-a8c9-2c0971435e06 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:25.524958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fd42a-7a08-4e3d-a8c9-2c0971435e06 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:25.525843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fd42a-7a08-4e3d-a8c9-2c0971435e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:25.557438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bc0fd42a-7a08-4e3d-a8c9-2c0971435e06 None None] MFA Rules not processed for user `551281a31ce24e2f9b81b36f69419311`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:25.559688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bc0fd42a-7a08-4e3d-a8c9-2c0971435e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:25.587403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:25.668682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 939/1902] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jul 28 04:00:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:25.691159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72aa85-a27b-43d6-9da2-b70c49efadd5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:25.694359 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cd72aa85-a27b-43d6-9da2-b70c49efadd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:25.743874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72aa85-a27b-43d6-9da2-b70c49efadd5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:25.760153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c3874-11e9-46a6-bbb2-f1c2702c765a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:25.760955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c3874-11e9-46a6-bbb2-f1c2702c765a service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:25.761290 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c3874-11e9-46a6-bbb2-f1c2702c765a service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:25.761641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c3874-11e9-46a6-bbb2-f1c2702c765a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:25.793720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c3874-11e9-46a6-bbb2-f1c2702c765a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:25.796159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c3874-11e9-46a6-bbb2-f1c2702c765a service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:25.812490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 964/1903] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:25.821508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fd42a-7a08-4e3d-a8c9-2c0971435e06 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:25.823950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bc0fd42a-7a08-4e3d-a8c9-2c0971435e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:25.831703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c3874-11e9-46a6-bbb2-f1c2702c765a service placement] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:25.833142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-464c3874-11e9-46a6-bbb2-f1c2702c765a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:25.854206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fd42a-7a08-4e3d-a8c9-2c0971435e06 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:25.864479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:25.866213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:25.866608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:25.867051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] PATH_INFO: `/v3/projects/f78ad65de49648d98a22b3e0eb083a35` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:25.868390 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f78ad65de49648d98a22b3e0eb083a35)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:25.883486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c3874-11e9-46a6-bbb2-f1c2702c765a service placement] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:25.886999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:25.908997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:25.909840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:25.910282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:25.910696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] PATH_INFO: `/v3/projects/e26d0a638fa2418a834defc95b0f4c08` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:25.912818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e26d0a638fa2418a834defc95b0f4c08)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:25.918573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:26.003528 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:26.010949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e26d0a638fa2418a834defc95b0f4c08', 'request_id': 'req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.013351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f78ad65de49648d98a22b3e0eb083a35', 'request_id': 'req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.019869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e26d0a638fa2418a834defc95b0f4c08', 'request_id': 'req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.022119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f78ad65de49648d98a22b3e0eb083a35', 'request_id': 'req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.028396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e26d0a638fa2418a834defc95b0f4c08', 'request_id': 'req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.031496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f78ad65de49648d98a22b3e0eb083a35', 'request_id': 'req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.044414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] The token cache is being invalidate because project e26d0a638fa2418a834defc95b0f4c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:26.044707 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.045557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] The token cache is being invalidate because project f78ad65de49648d98a22b3e0eb083a35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:26.045767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.045909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.046859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.048146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 965/1904] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:25 2022] DELETE /identity/v3/projects/e26d0a638fa2418a834defc95b0f4c08 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:26.049633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 940/1905] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:25 2022] DELETE /identity/v3/projects/f78ad65de49648d98a22b3e0eb083a35 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:26.053076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:26.054843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:26.056005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:26.056151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:26.131282 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b53b9f-17a0-470b-9c2a-3ffd55bf131d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:26.134407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc7b65-7189-4cd8-ae95-2b406a84cd5d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:26.136235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:26.136973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:26.137429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:26.137995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] PATH_INFO: `/v3/users/97f1a9a2217047c6bc95b5ba5ea4af3e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:26.139256 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f1a9a2217047c6bc95b5ba5ea4af3e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:26.139927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:26.140915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:26.141269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:26.141388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:26.141665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] PATH_INFO: `/v3/projects/8e121b4ad34c463b97cd2f7a631719b8` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:26.143543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e121b4ad34c463b97cd2f7a631719b8)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:26.146686 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:26.197889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:26.201191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f1a9a2217047c6bc95b5ba5ea4af3e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.202316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f1a9a2217047c6bc95b5ba5ea4af3e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.205443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e121b4ad34c463b97cd2f7a631719b8', 'request_id': 'req-3dd19e2d-016d-45e8-86ea-05f90952a78f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.212794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f1a9a2217047c6bc95b5ba5ea4af3e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.214256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e121b4ad34c463b97cd2f7a631719b8', 'request_id': 'req-3dd19e2d-016d-45e8-86ea-05f90952a78f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.223858 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e121b4ad34c463b97cd2f7a631719b8', 'request_id': 'req-3dd19e2d-016d-45e8-86ea-05f90952a78f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.227113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f1a9a2217047c6bc95b5ba5ea4af3e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.236699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f1a9a2217047c6bc95b5ba5ea4af3e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.239584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] The token cache is being invalidate because project 8e121b4ad34c463b97cd2f7a631719b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:26.240029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.240695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.242466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 966/1906] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:26 2022] DELETE /identity/v3/projects/8e121b4ad34c463b97cd2f7a631719b8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:26.243455 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:26.244372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 941/1907] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:26 2022] DELETE /identity/v3/users/97f1a9a2217047c6bc95b5ba5ea4af3e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:26.245308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:26.251815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:26.254729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:26.347268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd19e2d-016d-45e8-86ea-05f90952a78f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:26.354711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:26.355408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:26.355696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:26.355978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] PATH_INFO: `/v3/users/5de06169fbc645718d46abcccc374b8b` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:26.357508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de06169fbc645718d46abcccc374b8b)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:26.360873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:26.364607 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:26.367624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9db6c-f064-4036-b84c-1d1a9e5705fe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:26.382714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:26.383696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:26.384404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:26.385196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] PATH_INFO: `/v3/users/73924a1092d14885939cf0137b012437` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:26.387006 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73924a1092d14885939cf0137b012437)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:26.390172 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:26.438453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de06169fbc645718d46abcccc374b8b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.440208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de06169fbc645718d46abcccc374b8b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.500091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de06169fbc645718d46abcccc374b8b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.507061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73924a1092d14885939cf0137b012437'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.508086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73924a1092d14885939cf0137b012437'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.515936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de06169fbc645718d46abcccc374b8b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.526490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73924a1092d14885939cf0137b012437'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.530140 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de06169fbc645718d46abcccc374b8b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.542397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 967/1908] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:26 2022] DELETE /identity/v3/users/5de06169fbc645718d46abcccc374b8b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:26.559272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73924a1092d14885939cf0137b012437'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.575314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73924a1092d14885939cf0137b012437'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:26.589499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 942/1909] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:26 2022] DELETE /identity/v3/users/73924a1092d14885939cf0137b012437 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:27.184723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:27.186511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:27.216621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:27.218132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:27.372294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:27.412439 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec528580-8afa-4d94-993d-5e7b91bb611d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:27.423653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:27.424316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:27.424601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:27.424876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] PATH_INFO: `/v3/projects/97db4c5972fd4285b8fee56651504f04` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:27.426315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97db4c5972fd4285b8fee56651504f04)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:27.428657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:27.444625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b27f5-c022-4a36-81ba-014ee4dead61 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:27.449796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:27.450546 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:27.450729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:27.451055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] PATH_INFO: `/v3/projects/86986f85e3224f8a8ffb52a300a89a63` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:27.457154 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86986f85e3224f8a8ffb52a300a89a63)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:27.459268 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:27.514592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97db4c5972fd4285b8fee56651504f04', 'request_id': 'req-f622cafd-ef9e-4769-a33d-faebd69450a4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.522026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97db4c5972fd4285b8fee56651504f04', 'request_id': 'req-f622cafd-ef9e-4769-a33d-faebd69450a4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.523859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86986f85e3224f8a8ffb52a300a89a63', 'request_id': 'req-31383389-55f4-4e26-8574-29780b6e69a6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.529890 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97db4c5972fd4285b8fee56651504f04', 'request_id': 'req-f622cafd-ef9e-4769-a33d-faebd69450a4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.531792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86986f85e3224f8a8ffb52a300a89a63', 'request_id': 'req-31383389-55f4-4e26-8574-29780b6e69a6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.548544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86986f85e3224f8a8ffb52a300a89a63', 'request_id': 'req-31383389-55f4-4e26-8574-29780b6e69a6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.549750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] The token cache is being invalidate because project 97db4c5972fd4285b8fee56651504f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:27.550088 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.551004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.553077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 968/1910] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:27 2022] DELETE /identity/v3/projects/97db4c5972fd4285b8fee56651504f04 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:27.554109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:27.556126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:27.570071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] The token cache is being invalidate because project 86986f85e3224f8a8ffb52a300a89a63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:27.571637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.572643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31383389-55f4-4e26-8574-29780b6e69a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.578117 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:27.579967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 943/1911] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:27 2022] DELETE /identity/v3/projects/86986f85e3224f8a8ffb52a300a89a63 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:27.582213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d668c0f-e715-4c46-8b11-813f18ac4140 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:27.582583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d668c0f-e715-4c46-8b11-813f18ac4140 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:27.583478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d668c0f-e715-4c46-8b11-813f18ac4140 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:27.585372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 944/1912] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jul 28 04:00:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 04:00:27.586349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d668c0f-e715-4c46-8b11-813f18ac4140 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:27.588350 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7d668c0f-e715-4c46-8b11-813f18ac4140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:27.679907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622cafd-ef9e-4769-a33d-faebd69450a4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:27.682256 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d668c0f-e715-4c46-8b11-813f18ac4140 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:27.688269 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:27.689008 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:27.689298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:27.689651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] PATH_INFO: `/v3/users/a86c1febcbbe49a0881dfb9334cedba2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:27.692109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a86c1febcbbe49a0881dfb9334cedba2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:27.693796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:27.702468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:27.703706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:27.704189 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:27.704339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] PATH_INFO: `/v3/projects/aff9b02e532e4298abbfadc60a4122f7` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:27.710772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aff9b02e532e4298abbfadc60a4122f7)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:27.713935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:27.761853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a86c1febcbbe49a0881dfb9334cedba2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.762671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a86c1febcbbe49a0881dfb9334cedba2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.772517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a86c1febcbbe49a0881dfb9334cedba2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.774677 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aff9b02e532e4298abbfadc60a4122f7', 'request_id': 'req-6cfac7df-6831-48fc-b5f0-692ce7937f38'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.780721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aff9b02e532e4298abbfadc60a4122f7', 'request_id': 'req-6cfac7df-6831-48fc-b5f0-692ce7937f38'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.788630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aff9b02e532e4298abbfadc60a4122f7', 'request_id': 'req-6cfac7df-6831-48fc-b5f0-692ce7937f38'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.799978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a86c1febcbbe49a0881dfb9334cedba2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.803214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:27.805257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] The token cache is being invalidate because project aff9b02e532e4298abbfadc60a4122f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:27.805595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.806885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6cfac7df-6831-48fc-b5f0-692ce7937f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.809166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 969/1913] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:27 2022] DELETE /identity/v3/projects/aff9b02e532e4298abbfadc60a4122f7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:27.809304 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a86c1febcbbe49a0881dfb9334cedba2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:27.812111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1b26e-5017-47b6-991f-76d1942e6193 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:27.813116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1b26e-5017-47b6-991f-76d1942e6193 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:27.814165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1b26e-5017-47b6-991f-76d1942e6193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:27.820976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 945/1914] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:27 2022] DELETE /identity/v3/users/a86c1febcbbe49a0881dfb9334cedba2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:27.821878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:27.824069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:27.850903 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-89b1b26e-5017-47b6-991f-76d1942e6193 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:27.852175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-89b1b26e-5017-47b6-991f-76d1942e6193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:27.931820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfa37e-9ddf-4f5d-86bc-e3ec40b28aef admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:27.939800 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 970/1915] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:27.941416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37a2be-b144-4d07-824e-2b16968b1747 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:27.941934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37a2be-b144-4d07-824e-2b16968b1747 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:27.942405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37a2be-b144-4d07-824e-2b16968b1747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:27.943892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22d830-b567-4459-80c1-9e453f9c02e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:27.944584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22d830-b567-4459-80c1-9e453f9c02e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:27.944867 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22d830-b567-4459-80c1-9e453f9c02e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:27.947333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22d830-b567-4459-80c1-9e453f9c02e6 admin admin] PATH_INFO: `/v3/users/e1203d3ff9f047e9b4f44572bcff3698` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:27.949107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22d830-b567-4459-80c1-9e453f9c02e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1203d3ff9f047e9b4f44572bcff3698)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:27.951548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22d830-b567-4459-80c1-9e453f9c02e6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:27.965265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4d37a2be-b144-4d07-824e-2b16968b1747 None None] MFA Rules not processed for user `551281a31ce24e2f9b81b36f69419311`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:27.966864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4d37a2be-b144-4d07-824e-2b16968b1747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:28.019012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6d22d830-b567-4459-80c1-9e453f9c02e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1203d3ff9f047e9b4f44572bcff3698'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.020668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6d22d830-b567-4459-80c1-9e453f9c02e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1203d3ff9f047e9b4f44572bcff3698'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.035750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:28.040077 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6d22d830-b567-4459-80c1-9e453f9c02e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1203d3ff9f047e9b4f44572bcff3698'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.065508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6d22d830-b567-4459-80c1-9e453f9c02e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1203d3ff9f047e9b4f44572bcff3698'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.074256 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6d22d830-b567-4459-80c1-9e453f9c02e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1203d3ff9f047e9b4f44572bcff3698'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.092003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 971/1916] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jul 28 04:00:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:28.093795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cd43b-e01b-4cba-8263-8e2746cae741 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:28.094399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cd43b-e01b-4cba-8263-8e2746cae741 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:28.094687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cd43b-e01b-4cba-8263-8e2746cae741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:28.098884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 946/1917] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:27 2022] DELETE /identity/v3/users/e1203d3ff9f047e9b4f44572bcff3698 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:28.100288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05e7f0-77db-4b16-a70b-624a468054cc None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:28.100579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05e7f0-77db-4b16-a70b-624a468054cc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:28.100854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05e7f0-77db-4b16-a70b-624a468054cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:28.124566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ea9cd43b-e01b-4cba-8263-8e2746cae741 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:28.126281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ea9cd43b-e01b-4cba-8263-8e2746cae741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:28.181102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8a05e7f0-77db-4b16-a70b-624a468054cc None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:28.186678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8a05e7f0-77db-4b16-a70b-624a468054cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:28.253611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 972/1918] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:28.254478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cd43b-e01b-4cba-8263-8e2746cae741 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:28.255968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ea9cd43b-e01b-4cba-8263-8e2746cae741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:28.307251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 947/1919] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:28.308795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05e7f0-77db-4b16-a70b-624a468054cc None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:28.310178 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8a05e7f0-77db-4b16-a70b-624a468054cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:28.324162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05e7f0-77db-4b16-a70b-624a468054cc None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:28.330864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:28.331633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:28.331962 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:28.332335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] PATH_INFO: `/v3/projects/6edd43bd2d9c4f3280cff25aa19d4fc5` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:28.333596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] RBAC: Authorizing `identity:delete_project(project_id=6edd43bd2d9c4f3280cff25aa19d4fc5)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:28.336091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:28.345212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:28.376163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cd43b-e01b-4cba-8263-8e2746cae741 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:28.383031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584e8ba-183a-435d-9c8e-8774a2ac731e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:28.383795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584e8ba-183a-435d-9c8e-8774a2ac731e service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:28.384266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584e8ba-183a-435d-9c8e-8774a2ac731e service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:28.385939 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584e8ba-183a-435d-9c8e-8774a2ac731e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:28.396269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584e8ba-183a-435d-9c8e-8774a2ac731e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:28.397879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584e8ba-183a-435d-9c8e-8774a2ac731e service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:28.414763 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 973/1920] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:28.418072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da8296-d241-48ca-aede-3bf1324d56c3 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:28.418072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da8296-d241-48ca-aede-3bf1324d56c3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:28.418072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da8296-d241-48ca-aede-3bf1324d56c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:28.432678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6edd43bd2d9c4f3280cff25aa19d4fc5', 'request_id': 'req-2fcda367-a6e5-4c79-a232-9c13aa783cda'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.459841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6edd43bd2d9c4f3280cff25aa19d4fc5', 'request_id': 'req-2fcda367-a6e5-4c79-a232-9c13aa783cda'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.467034 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c4da8296-d241-48ca-aede-3bf1324d56c3 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:28.470296 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c4da8296-d241-48ca-aede-3bf1324d56c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:28.479123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6edd43bd2d9c4f3280cff25aa19d4fc5', 'request_id': 'req-2fcda367-a6e5-4c79-a232-9c13aa783cda'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.500148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] The token cache is being invalidate because project 6edd43bd2d9c4f3280cff25aa19d4fc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:28.500460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.501310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.504731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 948/1921] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:28 2022] DELETE /identity/v3/projects/6edd43bd2d9c4f3280cff25aa19d4fc5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:28.507795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:28.510036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:28.565383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 974/1922] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:28.566180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da8296-d241-48ca-aede-3bf1324d56c3 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:28.567822 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c4da8296-d241-48ca-aede-3bf1324d56c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:28.582864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da8296-d241-48ca-aede-3bf1324d56c3 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:28.588305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:28.589148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:28.589493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:28.589842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] PATH_INFO: `/v3/users/5ad5416bce8f4390988d0ce1300181d1` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:28.591141 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad5416bce8f4390988d0ce1300181d1)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:28.593193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:28.599482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:28.607830 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcda367-a6e5-4c79-a232-9c13aa783cda admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:28.614666 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cacb6a-fdb8-46f3-b7c6-fc179fb76e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:28.615324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cacb6a-fdb8-46f3-b7c6-fc179fb76e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:28.615610 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cacb6a-fdb8-46f3-b7c6-fc179fb76e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:28.615889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cacb6a-fdb8-46f3-b7c6-fc179fb76e6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:28.616605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cacb6a-fdb8-46f3-b7c6-fc179fb76e6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:00:28.616893 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cacb6a-fdb8-46f3-b7c6-fc179fb76e6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:28.618477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cacb6a-fdb8-46f3-b7c6-fc179fb76e6a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:28.633250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 949/1923] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:00:28.634444 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cacb6a-fdb8-46f3-b7c6-fc179fb76e6a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:28.645371 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cacb6a-fdb8-46f3-b7c6-fc179fb76e6a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:28.652086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b1a65-cf22-4b60-993d-bbaa954db674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:28.652913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b1a65-cf22-4b60-993d-bbaa954db674 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:28.653264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b1a65-cf22-4b60-993d-bbaa954db674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:28.653678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b1a65-cf22-4b60-993d-bbaa954db674 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:28.654595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b1a65-cf22-4b60-993d-bbaa954db674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:00:28.654915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b1a65-cf22-4b60-993d-bbaa954db674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:28.657276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b1a65-cf22-4b60-993d-bbaa954db674 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:28.662823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad5416bce8f4390988d0ce1300181d1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.665305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad5416bce8f4390988d0ce1300181d1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.671255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 950/1924] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:00:28.672320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b1a65-cf22-4b60-993d-bbaa954db674 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:28.685253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b1a65-cf22-4b60-993d-bbaa954db674 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:28.692739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad5416bce8f4390988d0ce1300181d1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.700488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2f231-13be-4cb4-b9f1-a80f245c9b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:28.701314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2f231-13be-4cb4-b9f1-a80f245c9b34 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:28.701783 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2f231-13be-4cb4-b9f1-a80f245c9b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:28.702174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2f231-13be-4cb4-b9f1-a80f245c9b34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:28.704389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad5416bce8f4390988d0ce1300181d1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.706938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2f231-13be-4cb4-b9f1-a80f245c9b34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:28.709992 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2f231-13be-4cb4-b9f1-a80f245c9b34 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:28.716178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad5416bce8f4390988d0ce1300181d1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.723162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 975/1925] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:28 2022] DELETE /identity/v3/users/5ad5416bce8f4390988d0ce1300181d1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:28.726058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:28.726058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:28.745924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 951/1926] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:28 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:28.745924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2f231-13be-4cb4-b9f1-a80f245c9b34 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:28.748481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3ed2f231-13be-4cb4-b9f1-a80f245c9b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:28.800489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5923e-047a-48c4-bf85-5415b346a287 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:28.811286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:28.827546 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2f231-13be-4cb4-b9f1-a80f245c9b34 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:28.835701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:28.836458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:28.836845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:28.837219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] PATH_INFO: `/v3/projects/8e3789199cfc425b85691753b38f375a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:28.838490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e3789199cfc425b85691753b38f375a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:28.840843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:28.880366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916fd90-b6e3-44c6-b408-f2e145554f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:28.881122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916fd90-b6e3-44c6-b408-f2e145554f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:28.881952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916fd90-b6e3-44c6-b408-f2e145554f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:28.882485 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916fd90-b6e3-44c6-b408-f2e145554f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:28.884469 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916fd90-b6e3-44c6-b408-f2e145554f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:28.888042 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916fd90-b6e3-44c6-b408-f2e145554f30 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:28.917180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e3789199cfc425b85691753b38f375a', 'request_id': 'req-b44eb388-b65a-45d5-b50a-20d27989313c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.926485 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e3789199cfc425b85691753b38f375a', 'request_id': 'req-b44eb388-b65a-45d5-b50a-20d27989313c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.930302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 976/1927] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:28 2022] POST /identity/v3/projects => generated 375 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:28.931272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916fd90-b6e3-44c6-b408-f2e145554f30 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:28.933568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a916fd90-b6e3-44c6-b408-f2e145554f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:28.938316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e3789199cfc425b85691753b38f375a', 'request_id': 'req-b44eb388-b65a-45d5-b50a-20d27989313c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.951728 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:28.957046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] The token cache is being invalidate because project 8e3789199cfc425b85691753b38f375a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:28.957345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.958324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:28.965382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 952/1928] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:28 2022] DELETE /identity/v3/projects/8e3789199cfc425b85691753b38f375a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:28.965382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:28.965382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.021170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916fd90-b6e3-44c6-b408-f2e145554f30 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.026322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458049e-1faf-423e-85d7-2edef72d7aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.027127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458049e-1faf-423e-85d7-2edef72d7aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.027666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458049e-1faf-423e-85d7-2edef72d7aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.027941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458049e-1faf-423e-85d7-2edef72d7aa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.028620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458049e-1faf-423e-85d7-2edef72d7aa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.030556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458049e-1faf-423e-85d7-2edef72d7aa7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.048564 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eb388-b65a-45d5-b50a-20d27989313c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.054018 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1c0bf-30bb-485d-8c0c-2ae10b776ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.054642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1c0bf-30bb-485d-8c0c-2ae10b776ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.054973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1c0bf-30bb-485d-8c0c-2ae10b776ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.055254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1c0bf-30bb-485d-8c0c-2ae10b776ac9 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.056343 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1c0bf-30bb-485d-8c0c-2ae10b776ac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.058417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1c0bf-30bb-485d-8c0c-2ae10b776ac9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.058564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 977/1929] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:28 2022] POST /identity/v3/users => generated 290 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:29.059897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176965e-b99d-47f0-87ed-7f1b96cf5323 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.060186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176965e-b99d-47f0-87ed-7f1b96cf5323 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.060497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176965e-b99d-47f0-87ed-7f1b96cf5323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.083154 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 953/1930] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:28 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:29.084125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1c0bf-30bb-485d-8c0c-2ae10b776ac9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.086153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-84b1c0bf-30bb-485d-8c0c-2ae10b776ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.091941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d176965e-b99d-47f0-87ed-7f1b96cf5323 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:29.093246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d176965e-b99d-47f0-87ed-7f1b96cf5323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.100070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1c0bf-30bb-485d-8c0c-2ae10b776ac9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.105114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0eea4-01ef-4067-a992-38fed0df1709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.105306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 978/1931] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:29.106134 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0eea4-01ef-4067-a992-38fed0df1709 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.106513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0eea4-01ef-4067-a992-38fed0df1709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.106810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0eea4-01ef-4067-a992-38fed0df1709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.107612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0eea4-01ef-4067-a992-38fed0df1709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.110472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0eea4-01ef-4067-a992-38fed0df1709 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.111983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176965e-b99d-47f0-87ed-7f1b96cf5323 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.125333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 954/1932] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:29.125333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7936af6-4951-4f40-a5ac-0662abeb9fef None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.125333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7936af6-4951-4f40-a5ac-0662abeb9fef None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.125333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7936af6-4951-4f40-a5ac-0662abeb9fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.125932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176965e-b99d-47f0-87ed-7f1b96cf5323 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.130720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8ae1e-a448-496d-9f75-77a7dca28170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.131415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8ae1e-a448-496d-9f75-77a7dca28170 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.131769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8ae1e-a448-496d-9f75-77a7dca28170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.132113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8ae1e-a448-496d-9f75-77a7dca28170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.132893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8ae1e-a448-496d-9f75-77a7dca28170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.134571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8ae1e-a448-496d-9f75-77a7dca28170 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.145519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 979/1933] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:29.146651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8ae1e-a448-496d-9f75-77a7dca28170 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.147110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f7936af6-4951-4f40-a5ac-0662abeb9fef None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:29.148725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f7936af6-4951-4f40-a5ac-0662abeb9fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.158814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8ae1e-a448-496d-9f75-77a7dca28170 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.162696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 955/1934] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:29.163824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4ca10-e3e5-4629-98ec-70d0e92b9036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.163970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7936af6-4951-4f40-a5ac-0662abeb9fef None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.164543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4ca10-e3e5-4629-98ec-70d0e92b9036 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.164832 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4ca10-e3e5-4629-98ec-70d0e92b9036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.165107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4ca10-e3e5-4629-98ec-70d0e92b9036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.165947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4ca10-e3e5-4629-98ec-70d0e92b9036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.167576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4ca10-e3e5-4629-98ec-70d0e92b9036 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.176436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7936af6-4951-4f40-a5ac-0662abeb9fef None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.184344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 980/1935] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:29.184490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20747076-67e9-4a79-867a-0aea35c98f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.184490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20747076-67e9-4a79-867a-0aea35c98f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.184644 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4ca10-e3e5-4629-98ec-70d0e92b9036 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.184741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20747076-67e9-4a79-867a-0aea35c98f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.184741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20747076-67e9-4a79-867a-0aea35c98f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.184741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20747076-67e9-4a79-867a-0aea35c98f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.185959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20747076-67e9-4a79-867a-0aea35c98f8d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.200087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 956/1936] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:29.200291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4ca10-e3e5-4629-98ec-70d0e92b9036 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.200409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20747076-67e9-4a79-867a-0aea35c98f8d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.203642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-20747076-67e9-4a79-867a-0aea35c98f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.207324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb2a1c-0723-48f4-9138-550d5f22e5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.207964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb2a1c-0723-48f4-9138-550d5f22e5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.208297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb2a1c-0723-48f4-9138-550d5f22e5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.208579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb2a1c-0723-48f4-9138-550d5f22e5a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.209366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb2a1c-0723-48f4-9138-550d5f22e5a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:00:29.209594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb2a1c-0723-48f4-9138-550d5f22e5a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.212809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb2a1c-0723-48f4-9138-550d5f22e5a1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.217005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20747076-67e9-4a79-867a-0aea35c98f8d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.226921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.228554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.229521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.230036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] PATH_INFO: `/v3/projects/12b2e384a4bf49628b6c01abcdd6e66b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.231718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b2e384a4bf49628b6c01abcdd6e66b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.233993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.234849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 981/1937] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:00:29.235831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb2a1c-0723-48f4-9138-550d5f22e5a1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.252897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb2a1c-0723-48f4-9138-550d5f22e5a1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.259598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e98dc2-9ae9-4d03-9584-04b55ac516cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.260384 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98dc2-9ae9-4d03-9584-04b55ac516cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.260674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98dc2-9ae9-4d03-9584-04b55ac516cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.260950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98dc2-9ae9-4d03-9584-04b55ac516cd admin admin] PATH_INFO: `/v3/projects/adbc458c03aa4f7e8ccb87a40d4550e2/users/5f8980d9d23b45278541b7d3bf867aad/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.290175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e98dc2-9ae9-4d03-9584-04b55ac516cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=adbc458c03aa4f7e8ccb87a40d4550e2, user_id=5f8980d9d23b45278541b7d3bf867aad, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.292820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e98dc2-9ae9-4d03-9584-04b55ac516cd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.315238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 982/1938] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:29 2022] PUT /identity/v3/projects/adbc458c03aa4f7e8ccb87a40d4550e2/users/5f8980d9d23b45278541b7d3bf867aad/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:29.316384 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e98dc2-9ae9-4d03-9584-04b55ac516cd admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.324395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:29.334523 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b2e384a4bf49628b6c01abcdd6e66b', 'request_id': 'req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:29.340027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b2e384a4bf49628b6c01abcdd6e66b', 'request_id': 'req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:29.347156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b2e384a4bf49628b6c01abcdd6e66b', 'request_id': 'req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:29.360519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] The token cache is being invalidate because project 12b2e384a4bf49628b6c01abcdd6e66b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:29.360807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:29.361768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:29.363364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 957/1939] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:29 2022] DELETE /identity/v3/projects/12b2e384a4bf49628b6c01abcdd6e66b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:29.364112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.365426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.384565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e98dc2-9ae9-4d03-9584-04b55ac516cd admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.435288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:29.441126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425e7f5-adab-4c6c-bf9f-8198c7ad5551 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.446414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd947a2-5258-4490-87d6-27c62f83dba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.447294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd947a2-5258-4490-87d6-27c62f83dba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.447575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd947a2-5258-4490-87d6-27c62f83dba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.447848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd947a2-5258-4490-87d6-27c62f83dba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.448517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd947a2-5258-4490-87d6-27c62f83dba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.450573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd947a2-5258-4490-87d6-27c62f83dba8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.453509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b83ccd-a6a5-432d-b519-fc6f1a5fffb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.454520 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b83ccd-a6a5-432d-b519-fc6f1a5fffb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.454907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b83ccd-a6a5-432d-b519-fc6f1a5fffb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.455244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b83ccd-a6a5-432d-b519-fc6f1a5fffb7 admin admin] PATH_INFO: `/v3/projects/551ad8000fca49e181e661b108a43a8e/users/4bdd68b67d154cfcb02982799a47b2cc/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.471172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b83ccd-a6a5-432d-b519-fc6f1a5fffb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=551ad8000fca49e181e661b108a43a8e, user_id=4bdd68b67d154cfcb02982799a47b2cc, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.473948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b83ccd-a6a5-432d-b519-fc6f1a5fffb7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.489372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 958/1940] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:29.490834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20644a6e-147c-4b94-b0fe-8902bf1d9329 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.491297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20644a6e-147c-4b94-b0fe-8902bf1d9329 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.491878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20644a6e-147c-4b94-b0fe-8902bf1d9329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.509671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 983/1941] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:29 2022] PUT /identity/v3/projects/551ad8000fca49e181e661b108a43a8e/users/4bdd68b67d154cfcb02982799a47b2cc/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:29.509671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8941140-14f1-4793-9d74-cc2c4080a317 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.509671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8941140-14f1-4793-9d74-cc2c4080a317 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.509671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8941140-14f1-4793-9d74-cc2c4080a317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.534017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d8941140-14f1-4793-9d74-cc2c4080a317 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:29.535689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d8941140-14f1-4793-9d74-cc2c4080a317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.536257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-20644a6e-147c-4b94-b0fe-8902bf1d9329 None None] MFA Rules not processed for user `5f8980d9d23b45278541b7d3bf867aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:29.537578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-20644a6e-147c-4b94-b0fe-8902bf1d9329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.618486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 984/1942] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:29.619367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8941140-14f1-4793-9d74-cc2c4080a317 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.630241 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 959/1943] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:29.631644 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61721de-e64e-4d7f-8e3d-17cd1ea72813 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.632035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61721de-e64e-4d7f-8e3d-17cd1ea72813 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.632412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61721de-e64e-4d7f-8e3d-17cd1ea72813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.635876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8941140-14f1-4793-9d74-cc2c4080a317 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.643039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03d224-2250-43f3-a429-8cc8e914f89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.643820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03d224-2250-43f3-a429-8cc8e914f89c admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.644204 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03d224-2250-43f3-a429-8cc8e914f89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.644579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03d224-2250-43f3-a429-8cc8e914f89c admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.645817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03d224-2250-43f3-a429-8cc8e914f89c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.647939 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03d224-2250-43f3-a429-8cc8e914f89c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.672559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f61721de-e64e-4d7f-8e3d-17cd1ea72813 None None] MFA Rules not processed for user `4bdd68b67d154cfcb02982799a47b2cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:29.673885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f61721de-e64e-4d7f-8e3d-17cd1ea72813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.679894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 985/1944] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/users => generated 287 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:29.681041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb9152-b786-497f-b192-d2ae1c8107e7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.681336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb9152-b786-497f-b192-d2ae1c8107e7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.681617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb9152-b786-497f-b192-d2ae1c8107e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.704932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-60eb9152-b786-497f-b192-d2ae1c8107e7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:29.706295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-60eb9152-b786-497f-b192-d2ae1c8107e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.708857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:29.720676 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 986/1945] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:29.721692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb9152-b786-497f-b192-d2ae1c8107e7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.724791 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-60eb9152-b786-497f-b192-d2ae1c8107e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.737830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb9152-b786-497f-b192-d2ae1c8107e7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.743604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9f4c5-4f58-4fab-b9f9-ed263b202656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.744221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9f4c5-4f58-4fab-b9f9-ed263b202656 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.744502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9f4c5-4f58-4fab-b9f9-ed263b202656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.744767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9f4c5-4f58-4fab-b9f9-ed263b202656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.745737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9f4c5-4f58-4fab-b9f9-ed263b202656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.747959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9f4c5-4f58-4fab-b9f9-ed263b202656 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.754372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 960/1946] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:29.754372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61721de-e64e-4d7f-8e3d-17cd1ea72813 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.766196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61721de-e64e-4d7f-8e3d-17cd1ea72813 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.772665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8f01b-ec9f-4573-9df4-222c0e23337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.773287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8f01b-ec9f-4573-9df4-222c0e23337d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.773577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8f01b-ec9f-4573-9df4-222c0e23337d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.773854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8f01b-ec9f-4573-9df4-222c0e23337d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.774570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8f01b-ec9f-4573-9df4-222c0e23337d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.776216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8f01b-ec9f-4573-9df4-222c0e23337d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.785920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 987/1947] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:29.785920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9f4c5-4f58-4fab-b9f9-ed263b202656 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.788552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 961/1948] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:29.789246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8f01b-ec9f-4573-9df4-222c0e23337d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.792633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3aa8f01b-ec9f-4573-9df4-222c0e23337d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:29.806128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8f01b-ec9f-4573-9df4-222c0e23337d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.861565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9f4c5-4f58-4fab-b9f9-ed263b202656 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.867915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73d863-300a-44b9-9014-c3955a5f4cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.868699 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73d863-300a-44b9-9014-c3955a5f4cae admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.869058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73d863-300a-44b9-9014-c3955a5f4cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.869878 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73d863-300a-44b9-9014-c3955a5f4cae admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.870671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73d863-300a-44b9-9014-c3955a5f4cae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:00:29.871077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73d863-300a-44b9-9014-c3955a5f4cae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.872777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73d863-300a-44b9-9014-c3955a5f4cae admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.877208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c9636-3a0d-4283-be3a-13c44ee67b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.877910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c9636-3a0d-4283-be3a-13c44ee67b0c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.878259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c9636-3a0d-4283-be3a-13c44ee67b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.878601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c9636-3a0d-4283-be3a-13c44ee67b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.879380 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c9636-3a0d-4283-be3a-13c44ee67b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.881841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c9636-3a0d-4283-be3a-13c44ee67b0c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.889644 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 988/1949] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:00:29.889644 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73d863-300a-44b9-9014-c3955a5f4cae admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.898096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73d863-300a-44b9-9014-c3955a5f4cae admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.902958 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f61d6d-56fd-409b-b548-5e55861912ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.903794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f61d6d-56fd-409b-b548-5e55861912ba admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.903992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f61d6d-56fd-409b-b548-5e55861912ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.904333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f61d6d-56fd-409b-b548-5e55861912ba admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.905023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f61d6d-56fd-409b-b548-5e55861912ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:29.907978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f61d6d-56fd-409b-b548-5e55861912ba admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:29.919005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 962/1950] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:29.919005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c9636-3a0d-4283-be3a-13c44ee67b0c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.934577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:29.946154 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 989/1951] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/users => generated 297 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:29.947349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f61d6d-56fd-409b-b548-5e55861912ba admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:29.968166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f61d6d-56fd-409b-b548-5e55861912ba admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.991987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c9636-3a0d-4283-be3a-13c44ee67b0c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:29.996970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23388a-273d-4df3-8c8f-db855cf65674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:29.997679 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23388a-273d-4df3-8c8f-db855cf65674 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:29.998030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23388a-273d-4df3-8c8f-db855cf65674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:29.998375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23388a-273d-4df3-8c8f-db855cf65674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:29.999205 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23388a-273d-4df3-8c8f-db855cf65674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.001498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23388a-273d-4df3-8c8f-db855cf65674 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.012369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 963/1952] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:30.013261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23388a-273d-4df3-8c8f-db855cf65674 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.024439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23388a-273d-4df3-8c8f-db855cf65674 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.029798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11875d32-25ab-4090-8949-191ab1d7e21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.030416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11875d32-25ab-4090-8949-191ab1d7e21c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.030701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11875d32-25ab-4090-8949-191ab1d7e21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.031299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11875d32-25ab-4090-8949-191ab1d7e21c admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.032304 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11875d32-25ab-4090-8949-191ab1d7e21c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.034234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11875d32-25ab-4090-8949-191ab1d7e21c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.034633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ce00c-b9a2-4ecc-9d48-ea6dd83a6457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.035341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ce00c-b9a2-4ecc-9d48-ea6dd83a6457 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.035675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ce00c-b9a2-4ecc-9d48-ea6dd83a6457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.035957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ce00c-b9a2-4ecc-9d48-ea6dd83a6457 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.036764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ce00c-b9a2-4ecc-9d48-ea6dd83a6457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.039277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ce00c-b9a2-4ecc-9d48-ea6dd83a6457 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.057627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 964/1953] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:30 2022] POST /identity/v3/users => generated 294 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:30.059323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11875d32-25ab-4090-8949-191ab1d7e21c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.071255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11875d32-25ab-4090-8949-191ab1d7e21c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.072652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 990/1954] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:29 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:30.073767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ce00c-b9a2-4ecc-9d48-ea6dd83a6457 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.100754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:30.140069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b3f76-706d-4dd8-b0bd-ecf3006ed350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.141258 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b3f76-706d-4dd8-b0bd-ecf3006ed350 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.141597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b3f76-706d-4dd8-b0bd-ecf3006ed350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.141966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b3f76-706d-4dd8-b0bd-ecf3006ed350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.142734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b3f76-706d-4dd8-b0bd-ecf3006ed350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.144654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b3f76-706d-4dd8-b0bd-ecf3006ed350 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.149646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ce00c-b9a2-4ecc-9d48-ea6dd83a6457 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.154980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedd016-4529-4c08-82b5-dec98d830f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.155738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedd016-4529-4c08-82b5-dec98d830f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.156266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedd016-4529-4c08-82b5-dec98d830f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.156622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedd016-4529-4c08-82b5-dec98d830f23 admin admin] PATH_INFO: `/v3/projects/9ed8ce4c8a3442deacf65acc0c05dad3/users/820e385c27af49f08301be61fcde369b/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.160451 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 965/1955] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:30.161413 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b3f76-706d-4dd8-b0bd-ecf3006ed350 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.172752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dedd016-4529-4c08-82b5-dec98d830f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ed8ce4c8a3442deacf65acc0c05dad3, user_id=820e385c27af49f08301be61fcde369b, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.174487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b3f76-706d-4dd8-b0bd-ecf3006ed350 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.175628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dedd016-4529-4c08-82b5-dec98d830f23 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.180387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a400f21-bc04-47a1-ad56-2db796b0c8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.181096 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a400f21-bc04-47a1-ad56-2db796b0c8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.181419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a400f21-bc04-47a1-ad56-2db796b0c8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.181804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a400f21-bc04-47a1-ad56-2db796b0c8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.182503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a400f21-bc04-47a1-ad56-2db796b0c8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.184150 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a400f21-bc04-47a1-ad56-2db796b0c8e7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.197962 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 966/1957] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:30.197962 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a400f21-bc04-47a1-ad56-2db796b0c8e7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.201721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 991/1957] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:30 2022] PUT /identity/v3/projects/9ed8ce4c8a3442deacf65acc0c05dad3/users/820e385c27af49f08301be61fcde369b/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:30.204029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedd016-4529-4c08-82b5-dec98d830f23 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.212950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a400f21-bc04-47a1-ad56-2db796b0c8e7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.272773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67229b8-7c33-4710-8fc0-a127752a5cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.273314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedd016-4529-4c08-82b5-dec98d830f23 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.273585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67229b8-7c33-4710-8fc0-a127752a5cbb admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.273991 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67229b8-7c33-4710-8fc0-a127752a5cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.274815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67229b8-7c33-4710-8fc0-a127752a5cbb admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.274944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67229b8-7c33-4710-8fc0-a127752a5cbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.276814 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67229b8-7c33-4710-8fc0-a127752a5cbb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.278835 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7b746-2c69-47fb-aa07-ee51fcc3fe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.279912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7b746-2c69-47fb-aa07-ee51fcc3fe03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.279912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7b746-2c69-47fb-aa07-ee51fcc3fe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.280170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7b746-2c69-47fb-aa07-ee51fcc3fe03 admin admin] PATH_INFO: `/v3/projects/837611eb7f264a808838c614975516c0/users/83c1dc3db38e41b9be0ec45ae297add7/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.293055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:30.299265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7b746-2c69-47fb-aa07-ee51fcc3fe03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=837611eb7f264a808838c614975516c0, user_id=83c1dc3db38e41b9be0ec45ae297add7, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.301325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 967/1958] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:30 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:30.302740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e5322-acb1-4472-85fb-8e198887b57b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.303064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e5322-acb1-4472-85fb-8e198887b57b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.303325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e5322-acb1-4472-85fb-8e198887b57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.325420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2a7e5322-acb1-4472-85fb-8e198887b57b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:30.326171 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7b746-2c69-47fb-aa07-ee51fcc3fe03 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.328297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2a7e5322-acb1-4472-85fb-8e198887b57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:30.345315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 968/1959] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:30.345315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e5322-acb1-4472-85fb-8e198887b57b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.365002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e5322-acb1-4472-85fb-8e198887b57b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.368259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 992/1960] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:30 2022] PUT /identity/v3/projects/837611eb7f264a808838c614975516c0/users/83c1dc3db38e41b9be0ec45ae297add7/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:30.369339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7b746-2c69-47fb-aa07-ee51fcc3fe03 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.455587 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7b746-2c69-47fb-aa07-ee51fcc3fe03 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.455908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205bd76-bb03-4109-8648-e38eadcb9ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.456819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205bd76-bb03-4109-8648-e38eadcb9ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.457442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205bd76-bb03-4109-8648-e38eadcb9ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.457874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205bd76-bb03-4109-8648-e38eadcb9ea3 admin admin] PATH_INFO: `/v3/projects/f0cd5be4a02f41eaa8a6d3995991d2bf/users/46eaf8b9c11c41d0bbe86e92bccf3486/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.463059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f42688-aba9-4323-8ef4-34a46ef2fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.463926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f42688-aba9-4323-8ef4-34a46ef2fd27 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.464285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f42688-aba9-4323-8ef4-34a46ef2fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.464643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f42688-aba9-4323-8ef4-34a46ef2fd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.466181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f42688-aba9-4323-8ef4-34a46ef2fd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.469262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f42688-aba9-4323-8ef4-34a46ef2fd27 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.477179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205bd76-bb03-4109-8648-e38eadcb9ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0cd5be4a02f41eaa8a6d3995991d2bf, user_id=46eaf8b9c11c41d0bbe86e92bccf3486, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.480213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205bd76-bb03-4109-8648-e38eadcb9ea3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.484637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 993/1961] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:30.485772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f42688-aba9-4323-8ef4-34a46ef2fd27 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.487888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-71f42688-aba9-4323-8ef4-34a46ef2fd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:30.496590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 969/1962] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:30 2022] PUT /identity/v3/projects/f0cd5be4a02f41eaa8a6d3995991d2bf/users/46eaf8b9c11c41d0bbe86e92bccf3486/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:30.497373 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205bd76-bb03-4109-8648-e38eadcb9ea3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.517332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:30.569443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f42688-aba9-4323-8ef4-34a46ef2fd27 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.585285 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205bd76-bb03-4109-8648-e38eadcb9ea3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.596790 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63793f2c-3540-4a38-a5d4-c8410f98d7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.597440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63793f2c-3540-4a38-a5d4-c8410f98d7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.597737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63793f2c-3540-4a38-a5d4-c8410f98d7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.598018 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63793f2c-3540-4a38-a5d4-c8410f98d7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.599069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63793f2c-3540-4a38-a5d4-c8410f98d7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.601708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63793f2c-3540-4a38-a5d4-c8410f98d7a8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.618335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 970/1963] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:30.619710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63793f2c-3540-4a38-a5d4-c8410f98d7a8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.634832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63793f2c-3540-4a38-a5d4-c8410f98d7a8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.637789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bc2bd-0a6f-424d-acf2-101382350886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.638502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bc2bd-0a6f-424d-acf2-101382350886 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.638888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bc2bd-0a6f-424d-acf2-101382350886 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.639236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bc2bd-0a6f-424d-acf2-101382350886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.644711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50987ac-ca44-485e-8c65-7c9778f6e052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.645780 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50987ac-ca44-485e-8c65-7c9778f6e052 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.646574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50987ac-ca44-485e-8c65-7c9778f6e052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.647030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50987ac-ca44-485e-8c65-7c9778f6e052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.648608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50987ac-ca44-485e-8c65-7c9778f6e052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.650095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bc2bd-0a6f-424d-acf2-101382350886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.651520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50987ac-ca44-485e-8c65-7c9778f6e052 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.651834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bc2bd-0a6f-424d-acf2-101382350886 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.664606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 994/1964] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:30.665668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bc2bd-0a6f-424d-acf2-101382350886 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.672880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 971/1965] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:30.672880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50987ac-ca44-485e-8c65-7c9778f6e052 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.683474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bc2bd-0a6f-424d-acf2-101382350886 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.690040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9839a4d-ba22-43cd-988d-c400c43093dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.690954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9839a4d-ba22-43cd-988d-c400c43093dd admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.691127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9839a4d-ba22-43cd-988d-c400c43093dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.691413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9839a4d-ba22-43cd-988d-c400c43093dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.691910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50987ac-ca44-485e-8c65-7c9778f6e052 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.692117 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9839a4d-ba22-43cd-988d-c400c43093dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.694164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9839a4d-ba22-43cd-988d-c400c43093dd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.697986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4757131-cfdb-46ab-85b8-dcd5564071bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.699879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4757131-cfdb-46ab-85b8-dcd5564071bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.700606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4757131-cfdb-46ab-85b8-dcd5564071bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.700902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4757131-cfdb-46ab-85b8-dcd5564071bd admin admin] PATH_INFO: `/v3/projects/837611eb7f264a808838c614975516c0/users/83c1dc3db38e41b9be0ec45ae297add7/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.703255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4757131-cfdb-46ab-85b8-dcd5564071bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=837611eb7f264a808838c614975516c0, user_id=83c1dc3db38e41b9be0ec45ae297add7, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.706081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4757131-cfdb-46ab-85b8-dcd5564071bd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.708257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 995/1966] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:30.709708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9839a4d-ba22-43cd-988d-c400c43093dd admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.728352 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9839a4d-ba22-43cd-988d-c400c43093dd admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.738883 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 972/1967] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:30 2022] PUT /identity/v3/projects/837611eb7f264a808838c614975516c0/users/83c1dc3db38e41b9be0ec45ae297add7/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:30.740414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4757131-cfdb-46ab-85b8-dcd5564071bd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:30.770978 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:30.820723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47ad5e-3446-4b41-8ce0-d3da552f0580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.822509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47ad5e-3446-4b41-8ce0-d3da552f0580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.825318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47ad5e-3446-4b41-8ce0-d3da552f0580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.825716 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47ad5e-3446-4b41-8ce0-d3da552f0580 admin admin] PATH_INFO: `/v3/projects/af91a995733b436f88c2d39d8fa4c51e/users/ac18e6efe98443189d45efe1e7eac5aa/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.865410 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47ad5e-3446-4b41-8ce0-d3da552f0580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af91a995733b436f88c2d39d8fa4c51e, user_id=ac18e6efe98443189d45efe1e7eac5aa, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.868666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47ad5e-3446-4b41-8ce0-d3da552f0580 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.874186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4757131-cfdb-46ab-85b8-dcd5564071bd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:30.883579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59017b8a-d675-48b9-b58b-6ef0d64c9501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:30.884429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59017b8a-d675-48b9-b58b-6ef0d64c9501 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.884652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59017b8a-d675-48b9-b58b-6ef0d64c9501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.884935 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59017b8a-d675-48b9-b58b-6ef0d64c9501 admin admin] PATH_INFO: `/v3/projects/f0cd5be4a02f41eaa8a6d3995991d2bf/users/46eaf8b9c11c41d0bbe86e92bccf3486/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.887906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 996/1968] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:30 2022] PUT /identity/v3/projects/af91a995733b436f88c2d39d8fa4c51e/users/ac18e6efe98443189d45efe1e7eac5aa/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:30.903884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59017b8a-d675-48b9-b58b-6ef0d64c9501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0cd5be4a02f41eaa8a6d3995991d2bf, user_id=46eaf8b9c11c41d0bbe86e92bccf3486, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:30.912773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13511c6f-4c59-40b1-a16a-967bddcd1083 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.913347 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13511c6f-4c59-40b1-a16a-967bddcd1083 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.914147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13511c6f-4c59-40b1-a16a-967bddcd1083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.916399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59017b8a-d675-48b9-b58b-6ef0d64c9501 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:30.951798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 973/1969] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:30 2022] PUT /identity/v3/projects/f0cd5be4a02f41eaa8a6d3995991d2bf/users/46eaf8b9c11c41d0bbe86e92bccf3486/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:30.953094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd96f9a-7d85-42fd-8e47-3576e8405c34 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:30.953387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd96f9a-7d85-42fd-8e47-3576e8405c34 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:30.953704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd96f9a-7d85-42fd-8e47-3576e8405c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:30.992219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-13511c6f-4c59-40b1-a16a-967bddcd1083 None None] MFA Rules not processed for user `83c1dc3db38e41b9be0ec45ae297add7`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:30.995994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-13511c6f-4c59-40b1-a16a-967bddcd1083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:31.021812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2dd96f9a-7d85-42fd-8e47-3576e8405c34 None None] MFA Rules not processed for user `ac18e6efe98443189d45efe1e7eac5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:31.024882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2dd96f9a-7d85-42fd-8e47-3576e8405c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:31.116972 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 997/1970] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:31.118480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789cea4-fa97-4d7a-a8bb-2f52515d58b8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:31.118729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789cea4-fa97-4d7a-a8bb-2f52515d58b8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:31.118989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789cea4-fa97-4d7a-a8bb-2f52515d58b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:31.129533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 974/1971] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:31.131734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd96f9a-7d85-42fd-8e47-3576e8405c34 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:31.147567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:31.157581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1789cea4-fa97-4d7a-a8bb-2f52515d58b8 None None] MFA Rules not processed for user `46eaf8b9c11c41d0bbe86e92bccf3486`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:31.159927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1789cea4-fa97-4d7a-a8bb-2f52515d58b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:31.219415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd96f9a-7d85-42fd-8e47-3576e8405c34 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:31.226022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e9e66-e95f-4b69-81ce-e7aed15ac924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:31.226820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e9e66-e95f-4b69-81ce-e7aed15ac924 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:31.227873 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e9e66-e95f-4b69-81ce-e7aed15ac924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:31.228966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e9e66-e95f-4b69-81ce-e7aed15ac924 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:31.230331 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e9e66-e95f-4b69-81ce-e7aed15ac924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:31.235669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e9e66-e95f-4b69-81ce-e7aed15ac924 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:31.282582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 998/1972] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:31.283721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789cea4-fa97-4d7a-a8bb-2f52515d58b8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:31.285587 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1789cea4-fa97-4d7a-a8bb-2f52515d58b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:31.305446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 975/1973] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:31 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:31.307102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e9e66-e95f-4b69-81ce-e7aed15ac924 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:31.310393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fc0e9e66-e95f-4b69-81ce-e7aed15ac924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:31.341866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:31.378111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789cea4-fa97-4d7a-a8bb-2f52515d58b8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:31.382898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92130b7-de08-4c28-9f7a-9742af1a6010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:31.383583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92130b7-de08-4c28-9f7a-9742af1a6010 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:31.383876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92130b7-de08-4c28-9f7a-9742af1a6010 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:31.384259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92130b7-de08-4c28-9f7a-9742af1a6010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:31.388460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e9e66-e95f-4b69-81ce-e7aed15ac924 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:31.392966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75a73c-c201-4220-8778-6c8aa3813412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:31.393693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75a73c-c201-4220-8778-6c8aa3813412 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:31.394048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75a73c-c201-4220-8778-6c8aa3813412 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:31.394442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75a73c-c201-4220-8778-6c8aa3813412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:31.484871 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75a73c-c201-4220-8778-6c8aa3813412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:31.486334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75a73c-c201-4220-8778-6c8aa3813412 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:31.497182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92130b7-de08-4c28-9f7a-9742af1a6010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:31.497952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 976/1974] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:31.499306 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75a73c-c201-4220-8778-6c8aa3813412 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:31.512595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92130b7-de08-4c28-9f7a-9742af1a6010 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:31.553266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 999/1975] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:31.606747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:31.642521 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5db6f-654b-446b-860b-3b710fc1168c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:31.643020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5db6f-654b-446b-860b-3b710fc1168c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:31.643556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5db6f-654b-446b-860b-3b710fc1168c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:31.659837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75a73c-c201-4220-8778-6c8aa3813412 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:31.673448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843ecf4-abe8-4c0e-b0c3-553f7df19e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:31.674354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843ecf4-abe8-4c0e-b0c3-553f7df19e78 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:31.674641 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843ecf4-abe8-4c0e-b0c3-553f7df19e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:31.674939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843ecf4-abe8-4c0e-b0c3-553f7df19e78 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:31.678772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843ecf4-abe8-4c0e-b0c3-553f7df19e78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:31.680965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843ecf4-abe8-4c0e-b0c3-553f7df19e78 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:31.708972 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-79c5db6f-654b-446b-860b-3b710fc1168c None None] MFA Rules not processed for user `820e385c27af49f08301be61fcde369b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:31.710391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-79c5db6f-654b-446b-860b-3b710fc1168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:31.733772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 977/1976] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:31 2022] POST /identity/v3/users => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:31.735222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843ecf4-abe8-4c0e-b0c3-553f7df19e78 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:31.761224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843ecf4-abe8-4c0e-b0c3-553f7df19e78 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:31.774911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e8698-00c3-410e-880a-16f12dd4c6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:31.775551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8698-00c3-410e-880a-16f12dd4c6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:31.775831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8698-00c3-410e-880a-16f12dd4c6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:31.776105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8698-00c3-410e-880a-16f12dd4c6b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:31.776808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e8698-00c3-410e-880a-16f12dd4c6b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:31.782229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e8698-00c3-410e-880a-16f12dd4c6b5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:31.809924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 978/1977] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:31.814360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e8698-00c3-410e-880a-16f12dd4c6b5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:31.839569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e8698-00c3-410e-880a-16f12dd4c6b5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:31.878743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea93461-d794-4540-ba6a-4a759e81bd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:31.881706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea93461-d794-4540-ba6a-4a759e81bd54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:31.881706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea93461-d794-4540-ba6a-4a759e81bd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:31.881706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea93461-d794-4540-ba6a-4a759e81bd54 admin admin] PATH_INFO: `/v3/projects/2dc9542c456e471da5ea0e00a0cb6050/users/b2bd06f8b18947d89d0e1a8fcd20d869/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:31.907268 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea93461-d794-4540-ba6a-4a759e81bd54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dc9542c456e471da5ea0e00a0cb6050, user_id=b2bd06f8b18947d89d0e1a8fcd20d869, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:31.911335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea93461-d794-4540-ba6a-4a759e81bd54 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:31.930682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1000/1978] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:31.947536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 979/1979] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:31 2022] PUT /identity/v3/projects/2dc9542c456e471da5ea0e00a0cb6050/users/b2bd06f8b18947d89d0e1a8fcd20d869/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:31.949596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5db6f-654b-446b-860b-3b710fc1168c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:31.961567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea93461-d794-4540-ba6a-4a759e81bd54 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:31.963351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5db6f-654b-446b-860b-3b710fc1168c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:32.057993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea93461-d794-4540-ba6a-4a759e81bd54 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:32.063429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21075268-80a4-4e35-8f63-8baac94a9ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:32.064577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21075268-80a4-4e35-8f63-8baac94a9ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:32.065087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21075268-80a4-4e35-8f63-8baac94a9ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:32.065375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21075268-80a4-4e35-8f63-8baac94a9ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:32.066336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21075268-80a4-4e35-8f63-8baac94a9ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:32.068154 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21075268-80a4-4e35-8f63-8baac94a9ee2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:32.082834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 980/1980] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:32.084980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21075268-80a4-4e35-8f63-8baac94a9ee2 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:32.110148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21075268-80a4-4e35-8f63-8baac94a9ee2 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:32.128067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933d089-f9d4-4c60-8279-2e4a3295408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:32.128949 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933d089-f9d4-4c60-8279-2e4a3295408d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:32.129311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933d089-f9d4-4c60-8279-2e4a3295408d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:32.130007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933d089-f9d4-4c60-8279-2e4a3295408d admin admin] PATH_INFO: `/v3/projects/2dc9542c456e471da5ea0e00a0cb6050/users/b2bd06f8b18947d89d0e1a8fcd20d869/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:32.133126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933d089-f9d4-4c60-8279-2e4a3295408d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dc9542c456e471da5ea0e00a0cb6050, user_id=b2bd06f8b18947d89d0e1a8fcd20d869, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:32.138507 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933d089-f9d4-4c60-8279-2e4a3295408d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:32.168283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 981/1981] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:32 2022] PUT /identity/v3/projects/2dc9542c456e471da5ea0e00a0cb6050/users/b2bd06f8b18947d89d0e1a8fcd20d869/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:32.180879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dae0cc-dc4d-49cc-af56-f696b9344145 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:32.181395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dae0cc-dc4d-49cc-af56-f696b9344145 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:32.181849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dae0cc-dc4d-49cc-af56-f696b9344145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:32.183454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2404a-4f24-47c3-9f84-413875c461f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:32.184099 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2404a-4f24-47c3-9f84-413875c461f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:32.184632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2404a-4f24-47c3-9f84-413875c461f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:32.184896 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2404a-4f24-47c3-9f84-413875c461f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:32.211046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:32.250121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-38dae0cc-dc4d-49cc-af56-f696b9344145 None None] MFA Rules not processed for user `b2bd06f8b18947d89d0e1a8fcd20d869`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:32.260137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-38dae0cc-dc4d-49cc-af56-f696b9344145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:32.267322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2404a-4f24-47c3-9f84-413875c461f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:32.281380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2404a-4f24-47c3-9f84-413875c461f8 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:32.324202 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1001/1982] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:00:32.408566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 982/1983] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:00:32.439064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2404a-4f24-47c3-9f84-413875c461f8 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:32.448872 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2404a-4f24-47c3-9f84-413875c461f8 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:32.540933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b7f70-9eeb-442c-a049-176d0c050bdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:32.541580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b7f70-9eeb-442c-a049-176d0c050bdb service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:32.541864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b7f70-9eeb-442c-a049-176d0c050bdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:32.542143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b7f70-9eeb-442c-a049-176d0c050bdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:32.666369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b7f70-9eeb-442c-a049-176d0c050bdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:32.668416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b7f70-9eeb-442c-a049-176d0c050bdb service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:32.700188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1002/1984] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:33.266509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dae0cc-dc4d-49cc-af56-f696b9344145 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:33.286714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:33.429672 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dae0cc-dc4d-49cc-af56-f696b9344145 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:33.438694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:33.440164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:33.440806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:33.441391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] PATH_INFO: `/v3/users/5f8980d9d23b45278541b7d3bf867aad` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:33.443440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8980d9d23b45278541b7d3bf867aad)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:33.446456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:33.541700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8980d9d23b45278541b7d3bf867aad'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:33.543299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8980d9d23b45278541b7d3bf867aad'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:33.562971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8980d9d23b45278541b7d3bf867aad'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:33.582678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8980d9d23b45278541b7d3bf867aad'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:33.592655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8980d9d23b45278541b7d3bf867aad'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:33.605428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 983/1985] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:33 2022] DELETE /identity/v3/users/5f8980d9d23b45278541b7d3bf867aad => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:33.608262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3371d0-0c0f-4e35-9c43-ad4fd542dff1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:33.608552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3371d0-0c0f-4e35-9c43-ad4fd542dff1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:33.608973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3371d0-0c0f-4e35-9c43-ad4fd542dff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:33.659062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fa3371d0-0c0f-4e35-9c43-ad4fd542dff1 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:33.660310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fa3371d0-0c0f-4e35-9c43-ad4fd542dff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:33.778918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1003/1986] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:33.794252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:33.796566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:33.900733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3995515-e187-4a32-b53b-0b18270ae471 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:33.915150 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0b174-08d5-4207-ae7b-711b389dc789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:33.916010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0b174-08d5-4207-ae7b-711b389dc789 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:33.916366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0b174-08d5-4207-ae7b-711b389dc789 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:33.916806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0b174-08d5-4207-ae7b-711b389dc789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:33.929415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0b174-08d5-4207-ae7b-711b389dc789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:33.930999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0b174-08d5-4207-ae7b-711b389dc789 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:33.943866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 984/1987] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:34.097234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c76081-d0a4-4b88-8704-511272902d79 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:34.097688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c76081-d0a4-4b88-8704-511272902d79 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:34.097820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c76081-d0a4-4b88-8704-511272902d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:34.120764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-87c76081-d0a4-4b88-8704-511272902d79 None None] MFA Rules not processed for user `820e385c27af49f08301be61fcde369b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:34.122002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-87c76081-d0a4-4b88-8704-511272902d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:34.133060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:34.195523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1004/1988] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:34.476497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0b174-08d5-4207-ae7b-711b389dc789 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:34.478094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c8b0b174-08d5-4207-ae7b-711b389dc789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:34.523274 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0b174-08d5-4207-ae7b-711b389dc789 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:34.535339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:34.537067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:34.539034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:34.539034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] PATH_INFO: `/v3/projects/adbc458c03aa4f7e8ccb87a40d4550e2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:34.543177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adbc458c03aa4f7e8ccb87a40d4550e2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:34.548778 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:34.637453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:34.661881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adbc458c03aa4f7e8ccb87a40d4550e2', 'request_id': 'req-e8811037-9119-4ae2-af58-82ebe5a01d40'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:34.674359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adbc458c03aa4f7e8ccb87a40d4550e2', 'request_id': 'req-e8811037-9119-4ae2-af58-82ebe5a01d40'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:34.685901 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adbc458c03aa4f7e8ccb87a40d4550e2', 'request_id': 'req-e8811037-9119-4ae2-af58-82ebe5a01d40'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:34.701427 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] The token cache is being invalidate because project adbc458c03aa4f7e8ccb87a40d4550e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:34.701881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:34.704014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:34.706516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 985/1989] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:34 2022] DELETE /identity/v3/projects/adbc458c03aa4f7e8ccb87a40d4550e2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:34.710772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c76081-d0a4-4b88-8704-511272902d79 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:34.740687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c76081-d0a4-4b88-8704-511272902d79 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:34.842494 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b799e2d-f047-4526-9eac-94386d86695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:34.843853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b799e2d-f047-4526-9eac-94386d86695e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:34.844163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b799e2d-f047-4526-9eac-94386d86695e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:34.844640 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b799e2d-f047-4526-9eac-94386d86695e admin admin] PATH_INFO: `/v3/users/83c1dc3db38e41b9be0ec45ae297add7` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:34.846111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b799e2d-f047-4526-9eac-94386d86695e admin admin] RBAC: Authorizing `identity:delete_user(user_id=83c1dc3db38e41b9be0ec45ae297add7)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:34.848937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b799e2d-f047-4526-9eac-94386d86695e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:34.931892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9b799e2d-f047-4526-9eac-94386d86695e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83c1dc3db38e41b9be0ec45ae297add7'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:34.933091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9b799e2d-f047-4526-9eac-94386d86695e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83c1dc3db38e41b9be0ec45ae297add7'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:34.961524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9b799e2d-f047-4526-9eac-94386d86695e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83c1dc3db38e41b9be0ec45ae297add7'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:34.975533 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9b799e2d-f047-4526-9eac-94386d86695e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83c1dc3db38e41b9be0ec45ae297add7'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:34.985849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9b799e2d-f047-4526-9eac-94386d86695e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83c1dc3db38e41b9be0ec45ae297add7'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:34.993485 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1005/1990] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:34 2022] DELETE /identity/v3/users/83c1dc3db38e41b9be0ec45ae297add7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:35.377036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:35.378699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:35.393268 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8811037-9119-4ae2-af58-82ebe5a01d40 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:35.454972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:35.455721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:35.456007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:35.456286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] PATH_INFO: `/v3/projects/837611eb7f264a808838c614975516c0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:35.457466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=837611eb7f264a808838c614975516c0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:35.459410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:35.507408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '837611eb7f264a808838c614975516c0', 'request_id': 'req-3eecde1b-1e66-46cd-a980-eca89e9718c9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:35.512863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '837611eb7f264a808838c614975516c0', 'request_id': 'req-3eecde1b-1e66-46cd-a980-eca89e9718c9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:35.522294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '837611eb7f264a808838c614975516c0', 'request_id': 'req-3eecde1b-1e66-46cd-a980-eca89e9718c9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:35.541655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] The token cache is being invalidate because project 837611eb7f264a808838c614975516c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:35.542075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:35.542845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3eecde1b-1e66-46cd-a980-eca89e9718c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:35.549499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 986/1991] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:35 2022] DELETE /identity/v3/projects/837611eb7f264a808838c614975516c0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:35.551284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584f2b5-85fe-4d80-a805-6356913c7101 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:35.551579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584f2b5-85fe-4d80-a805-6356913c7101 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:35.551855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584f2b5-85fe-4d80-a805-6356913c7101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:35.574714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3584f2b5-85fe-4d80-a805-6356913c7101 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:35.575991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3584f2b5-85fe-4d80-a805-6356913c7101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:35.612764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:35.642478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1006/1992] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:35.650548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a661-e779-42a1-8f1a-50ea23ed973d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:35.650905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a661-e779-42a1-8f1a-50ea23ed973d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:35.651189 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a661-e779-42a1-8f1a-50ea23ed973d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:35.672814 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-07c7a661-e779-42a1-8f1a-50ea23ed973d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:35.674042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-07c7a661-e779-42a1-8f1a-50ea23ed973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:35.682240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 987/1993] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:35.686609 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584f2b5-85fe-4d80-a805-6356913c7101 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:35.696995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584f2b5-85fe-4d80-a805-6356913c7101 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:35.701833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbb956-6434-4d72-8afb-7325949464c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:35.702227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbb956-6434-4d72-8afb-7325949464c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:35.703074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbb956-6434-4d72-8afb-7325949464c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:35.703358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbb956-6434-4d72-8afb-7325949464c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:35.704066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbb956-6434-4d72-8afb-7325949464c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:00:35.704372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbb956-6434-4d72-8afb-7325949464c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:35.705991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbb956-6434-4d72-8afb-7325949464c7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:35.719144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1007/1994] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:00:35.723697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a661-e779-42a1-8f1a-50ea23ed973d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:35.733392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a661-e779-42a1-8f1a-50ea23ed973d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:35.738801 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7913097-0dda-4ff7-bca2-6062af7730c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:35.739441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7913097-0dda-4ff7-bca2-6062af7730c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:35.739728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7913097-0dda-4ff7-bca2-6062af7730c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:35.740001 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7913097-0dda-4ff7-bca2-6062af7730c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:35.740745 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7913097-0dda-4ff7-bca2-6062af7730c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:35.742649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7913097-0dda-4ff7-bca2-6062af7730c1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:35.776616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 988/1995] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:35 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:35.777475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbb956-6434-4d72-8afb-7325949464c7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:35.788477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbb956-6434-4d72-8afb-7325949464c7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:35.855430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a1f6-c957-41f0-9781-0718bc0abc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:35.855430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a1f6-c957-41f0-9781-0718bc0abc28 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:35.856092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a1f6-c957-41f0-9781-0718bc0abc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:35.856092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a1f6-c957-41f0-9781-0718bc0abc28 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:35.856871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349a1f6-c957-41f0-9781-0718bc0abc28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:35.858997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349a1f6-c957-41f0-9781-0718bc0abc28 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:35.883863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1008/1996] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:35 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:00:35.885082 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7913097-0dda-4ff7-bca2-6062af7730c1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:35.896332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7913097-0dda-4ff7-bca2-6062af7730c1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:35.901385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694a247-a220-472e-8447-e14d8f573221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:35.902138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694a247-a220-472e-8447-e14d8f573221 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:35.902491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694a247-a220-472e-8447-e14d8f573221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:35.902822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694a247-a220-472e-8447-e14d8f573221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:35.903529 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694a247-a220-472e-8447-e14d8f573221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:35.907224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694a247-a220-472e-8447-e14d8f573221 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:35.918803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 989/1997] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:00:35.923659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a1f6-c957-41f0-9781-0718bc0abc28 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:35.934043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a1f6-c957-41f0-9781-0718bc0abc28 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:35.938920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcad48-3763-41ca-bbcf-c8cb7ceb7b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:35.939541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcad48-3763-41ca-bbcf-c8cb7ceb7b77 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:35.939820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcad48-3763-41ca-bbcf-c8cb7ceb7b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:35.940095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcad48-3763-41ca-bbcf-c8cb7ceb7b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:35.940826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcad48-3763-41ca-bbcf-c8cb7ceb7b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:35.942463 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcad48-3763-41ca-bbcf-c8cb7ceb7b77 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:35.953913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1009/1998] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:35.958071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694a247-a220-472e-8447-e14d8f573221 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:35.969853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694a247-a220-472e-8447-e14d8f573221 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:35.974565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e695c-6d84-4ab3-a361-9dec521ce2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:35.975260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e695c-6d84-4ab3-a361-9dec521ce2c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:35.975548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e695c-6d84-4ab3-a361-9dec521ce2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:35.975831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e695c-6d84-4ab3-a361-9dec521ce2c7 admin admin] PATH_INFO: `/v3/projects/3e4289e377d24036b968d0ae71c61dcc/users/518ad5eb820e44aa9689215e6038a521/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:35.979675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:35.994188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e695c-6d84-4ab3-a361-9dec521ce2c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4289e377d24036b968d0ae71c61dcc, user_id=518ad5eb820e44aa9689215e6038a521, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:35.996983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e695c-6d84-4ab3-a361-9dec521ce2c7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:36.011933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 990/1999] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:35 2022] PUT /identity/v3/projects/3e4289e377d24036b968d0ae71c61dcc/users/518ad5eb820e44aa9689215e6038a521/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:36.019446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af549f4f-08e6-4a48-af33-88ff1ac0a058 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:36.019732 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af549f4f-08e6-4a48-af33-88ff1ac0a058 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:36.020038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af549f4f-08e6-4a48-af33-88ff1ac0a058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:36.040684 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-af549f4f-08e6-4a48-af33-88ff1ac0a058 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:36.041897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-af549f4f-08e6-4a48-af33-88ff1ac0a058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:36.106094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1010/2000] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:36.116091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e695c-6d84-4ab3-a361-9dec521ce2c7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:36.117787 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e23e695c-6d84-4ab3-a361-9dec521ce2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:36.174463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e695c-6d84-4ab3-a361-9dec521ce2c7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:36.178061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e92fe-b21c-41f5-b128-fd7a509f7751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:36.178677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e92fe-b21c-41f5-b128-fd7a509f7751 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:36.178995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e92fe-b21c-41f5-b128-fd7a509f7751 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:36.179273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e92fe-b21c-41f5-b128-fd7a509f7751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:36.188948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e92fe-b21c-41f5-b128-fd7a509f7751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:36.190383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e92fe-b21c-41f5-b128-fd7a509f7751 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:36.200048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 991/2001] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:37.176201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40320c7-fed4-4fe2-851e-463a17dc57ef None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:37.176635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40320c7-fed4-4fe2-851e-463a17dc57ef None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:37.176821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40320c7-fed4-4fe2-851e-463a17dc57ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:37.207918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e40320c7-fed4-4fe2-851e-463a17dc57ef None None] MFA Rules not processed for user `518ad5eb820e44aa9689215e6038a521`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:37.209161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e40320c7-fed4-4fe2-851e-463a17dc57ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:37.243823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:37.276813 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1011/2002] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:00:37.289020 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e92fe-b21c-41f5-b128-fd7a509f7751 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:37.290441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0d0e92fe-b21c-41f5-b128-fd7a509f7751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:37.355955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e92fe-b21c-41f5-b128-fd7a509f7751 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:37.359812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ab94b-a5b1-4e21-9636-461ab6dee7b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:37.360430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab94b-a5b1-4e21-9636-461ab6dee7b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:37.360716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab94b-a5b1-4e21-9636-461ab6dee7b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:37.360993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab94b-a5b1-4e21-9636-461ab6dee7b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:37.370269 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ab94b-a5b1-4e21-9636-461ab6dee7b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:37.371741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ab94b-a5b1-4e21-9636-461ab6dee7b0 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:37.381105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 992/2003] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:37.868428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40320c7-fed4-4fe2-851e-463a17dc57ef None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:37.869834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e40320c7-fed4-4fe2-851e-463a17dc57ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:37.882238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40320c7-fed4-4fe2-851e-463a17dc57ef None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:37.885812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb6c20-7b4f-4c68-abfc-ca103c7509c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:37.886426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb6c20-7b4f-4c68-abfc-ca103c7509c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:37.886711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb6c20-7b4f-4c68-abfc-ca103c7509c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:37.887039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb6c20-7b4f-4c68-abfc-ca103c7509c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:37.888783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-15bb6c20-7b4f-4c68-abfc-ca103c7509c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:37.960107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb6c20-7b4f-4c68-abfc-ca103c7509c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:37.961563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb6c20-7b4f-4c68-abfc-ca103c7509c3 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:37.972218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1012/2004] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:38.743244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128ffde-73d8-457c-8c17-7b09e5a2e3cf None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:38.743739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128ffde-73d8-457c-8c17-7b09e5a2e3cf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:38.743864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128ffde-73d8-457c-8c17-7b09e5a2e3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:38.763529 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5128ffde-73d8-457c-8c17-7b09e5a2e3cf None None] MFA Rules not processed for user `518ad5eb820e44aa9689215e6038a521`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:38.764738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5128ffde-73d8-457c-8c17-7b09e5a2e3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:38.771796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 993/2005] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:00:39.448500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb6c20-7b4f-4c68-abfc-ca103c7509c3 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:39.461917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb6c20-7b4f-4c68-abfc-ca103c7509c3 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:39.482424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7c8a0-d3ed-4610-9263-d6f570ab73d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:39.483096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7c8a0-d3ed-4610-9263-d6f570ab73d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:39.483865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7c8a0-d3ed-4610-9263-d6f570ab73d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:39.484273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7c8a0-d3ed-4610-9263-d6f570ab73d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:39.486338 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bed7c8a0-d3ed-4610-9263-d6f570ab73d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:39.567450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7c8a0-d3ed-4610-9263-d6f570ab73d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:39.569076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7c8a0-d3ed-4610-9263-d6f570ab73d4 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:39.585020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1013/2006] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:48.757218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:49.576081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=71 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:49.885663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128ffde-73d8-457c-8c17-7b09e5a2e3cf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:49.887233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5128ffde-73d8-457c-8c17-7b09e5a2e3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:49.901321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128ffde-73d8-457c-8c17-7b09e5a2e3cf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:49.906272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:49.906961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:49.907242 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:49.907516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] PATH_INFO: `/v3/users/ac18e6efe98443189d45efe1e7eac5aa` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:49.909131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac18e6efe98443189d45efe1e7eac5aa)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:49.910714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:49.963498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac18e6efe98443189d45efe1e7eac5aa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:49.964435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac18e6efe98443189d45efe1e7eac5aa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:49.972480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac18e6efe98443189d45efe1e7eac5aa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:49.988423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac18e6efe98443189d45efe1e7eac5aa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:49.996961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac18e6efe98443189d45efe1e7eac5aa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:50.004545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 994/2007] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:49 2022] DELETE /identity/v3/users/ac18e6efe98443189d45efe1e7eac5aa => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:50.009111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0172b-4653-4126-a3a8-aae7aa848935 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:50.009401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0172b-4653-4126-a3a8-aae7aa848935 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:50.009704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0172b-4653-4126-a3a8-aae7aa848935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:50.029445 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5ac0172b-4653-4126-a3a8-aae7aa848935 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:50.031321 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5ac0172b-4653-4126-a3a8-aae7aa848935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:50.094989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1014/2008] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:50.104352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:50.105888 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:50.175910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e91ec7-d604-4667-ba27-a0eab123ed86 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:50.179575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f4d2b-bfc3-4205-82d6-9025579211df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:50.180191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f4d2b-bfc3-4205-82d6-9025579211df service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:50.180473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f4d2b-bfc3-4205-82d6-9025579211df service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:50.180746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f4d2b-bfc3-4205-82d6-9025579211df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:50.190400 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f4d2b-bfc3-4205-82d6-9025579211df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:50.191875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f4d2b-bfc3-4205-82d6-9025579211df service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:50.201434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 995/2009] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:50.524253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0172b-4653-4126-a3a8-aae7aa848935 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:50.526035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5ac0172b-4653-4126-a3a8-aae7aa848935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:50.572263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0172b-4653-4126-a3a8-aae7aa848935 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:50.580825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:50.581559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:50.581912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:50.582259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] PATH_INFO: `/v3/projects/af91a995733b436f88c2d39d8fa4c51e` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:50.584003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af91a995733b436f88c2d39d8fa4c51e)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:50.586020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:50.661997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af91a995733b436f88c2d39d8fa4c51e', 'request_id': 'req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:50.668937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af91a995733b436f88c2d39d8fa4c51e', 'request_id': 'req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:50.675826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af91a995733b436f88c2d39d8fa4c51e', 'request_id': 'req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:50.691671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] The token cache is being invalidate because project af91a995733b436f88c2d39d8fa4c51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:50.692325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:50.693080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:50.695208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1015/2010] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:50 2022] DELETE /identity/v3/projects/af91a995733b436f88c2d39d8fa4c51e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:50.698672 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f4d2b-bfc3-4205-82d6-9025579211df service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:50.708905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f4d2b-bfc3-4205-82d6-9025579211df service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:50.771539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd53607-0828-46ac-b368-ee89db9a7393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:50.772397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd53607-0828-46ac-b368-ee89db9a7393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:50.772986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd53607-0828-46ac-b368-ee89db9a7393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:50.773097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd53607-0828-46ac-b368-ee89db9a7393 admin admin] PATH_INFO: `/v3/users/b2bd06f8b18947d89d0e1a8fcd20d869` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:50.787048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd53607-0828-46ac-b368-ee89db9a7393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2bd06f8b18947d89d0e1a8fcd20d869)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:50.787048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd53607-0828-46ac-b368-ee89db9a7393 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:50.804541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:50.837705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3dd53607-0828-46ac-b368-ee89db9a7393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2bd06f8b18947d89d0e1a8fcd20d869'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:50.838363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3dd53607-0828-46ac-b368-ee89db9a7393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2bd06f8b18947d89d0e1a8fcd20d869'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:50.846054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3dd53607-0828-46ac-b368-ee89db9a7393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2bd06f8b18947d89d0e1a8fcd20d869'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:50.861097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3dd53607-0828-46ac-b368-ee89db9a7393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2bd06f8b18947d89d0e1a8fcd20d869'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:50.869575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3dd53607-0828-46ac-b368-ee89db9a7393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2bd06f8b18947d89d0e1a8fcd20d869'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:50.876226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 996/2011] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:50 2022] DELETE /identity/v3/users/b2bd06f8b18947d89d0e1a8fcd20d869 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:51.166120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:51.168728 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:51.204989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760c6bf-635b-4a7b-a9bf-f41e5fd1a672 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:51.266051 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:51.266708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:51.267038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:51.267316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] PATH_INFO: `/v3/projects/2dc9542c456e471da5ea0e00a0cb6050` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:51.269204 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dc9542c456e471da5ea0e00a0cb6050)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:51.271351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:51.303657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:51.328078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dc9542c456e471da5ea0e00a0cb6050', 'request_id': 'req-c109a301-04a4-42a1-a218-b4e70e47e975'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:51.333765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dc9542c456e471da5ea0e00a0cb6050', 'request_id': 'req-c109a301-04a4-42a1-a218-b4e70e47e975'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:51.339816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dc9542c456e471da5ea0e00a0cb6050', 'request_id': 'req-c109a301-04a4-42a1-a218-b4e70e47e975'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:51.352553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] The token cache is being invalidate because project 2dc9542c456e471da5ea0e00a0cb6050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:51.352945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:51.353573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c109a301-04a4-42a1-a218-b4e70e47e975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:51.355116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1016/2012] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:51 2022] DELETE /identity/v3/projects/2dc9542c456e471da5ea0e00a0cb6050 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:51.361653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243741-144a-4ba8-8d52-a0d4615f9b22 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:51.362126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243741-144a-4ba8-8d52-a0d4615f9b22 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:51.362214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243741-144a-4ba8-8d52-a0d4615f9b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:51.381701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2e243741-144a-4ba8-8d52-a0d4615f9b22 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:51.382949 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2e243741-144a-4ba8-8d52-a0d4615f9b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:51.439704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 997/2013] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:51.448177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eb16a-a687-402e-94c9-ef56f7a6447c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:51.448467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eb16a-a687-402e-94c9-ef56f7a6447c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:51.448746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eb16a-a687-402e-94c9-ef56f7a6447c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:51.469074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-569eb16a-a687-402e-94c9-ef56f7a6447c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:51.470255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-569eb16a-a687-402e-94c9-ef56f7a6447c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:51.478055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1017/2014] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:51.482102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243741-144a-4ba8-8d52-a0d4615f9b22 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:51.491552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243741-144a-4ba8-8d52-a0d4615f9b22 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:51.496067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970aa1b-867f-435b-9f0f-505ff273c750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:51.496669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970aa1b-867f-435b-9f0f-505ff273c750 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:51.496949 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970aa1b-867f-435b-9f0f-505ff273c750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:51.497224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970aa1b-867f-435b-9f0f-505ff273c750 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:51.497904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970aa1b-867f-435b-9f0f-505ff273c750 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:00:51.498207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970aa1b-867f-435b-9f0f-505ff273c750 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:51.500359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970aa1b-867f-435b-9f0f-505ff273c750 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:51.511757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 998/2015] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:00:51.516738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569eb16a-a687-402e-94c9-ef56f7a6447c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:51.526201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569eb16a-a687-402e-94c9-ef56f7a6447c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:51.531291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920a8ae-ec32-4716-8357-eec4c9119ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:51.531901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920a8ae-ec32-4716-8357-eec4c9119ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:51.532192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920a8ae-ec32-4716-8357-eec4c9119ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:51.532465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920a8ae-ec32-4716-8357-eec4c9119ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:51.533137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920a8ae-ec32-4716-8357-eec4c9119ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:51.535062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920a8ae-ec32-4716-8357-eec4c9119ff4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:51.563715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1018/2016] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:51 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:51.567817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970aa1b-867f-435b-9f0f-505ff273c750 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:51.578503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970aa1b-867f-435b-9f0f-505ff273c750 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:51.633088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3645700-f4f7-4c31-8784-f1d85b9b7b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:51.633702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3645700-f4f7-4c31-8784-f1d85b9b7b28 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:51.633978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3645700-f4f7-4c31-8784-f1d85b9b7b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:51.634251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3645700-f4f7-4c31-8784-f1d85b9b7b28 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:51.634964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3645700-f4f7-4c31-8784-f1d85b9b7b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:51.636860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3645700-f4f7-4c31-8784-f1d85b9b7b28 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:51.658270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 999/2017] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:51 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:00:51.662527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920a8ae-ec32-4716-8357-eec4c9119ff4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:51.673188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920a8ae-ec32-4716-8357-eec4c9119ff4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:51.677786 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564145cf-ecc7-46bd-a816-b64568137506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:51.678390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564145cf-ecc7-46bd-a816-b64568137506 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:51.678672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564145cf-ecc7-46bd-a816-b64568137506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:51.678981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564145cf-ecc7-46bd-a816-b64568137506 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:51.679685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564145cf-ecc7-46bd-a816-b64568137506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:51.681302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564145cf-ecc7-46bd-a816-b64568137506 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:51.691564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1019/2018] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:51.695305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3645700-f4f7-4c31-8784-f1d85b9b7b28 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:51.706699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3645700-f4f7-4c31-8784-f1d85b9b7b28 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:51.711616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e269472-e7bb-44f3-8d96-6cb0eb9cabbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:51.712233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e269472-e7bb-44f3-8d96-6cb0eb9cabbd admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:51.712548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e269472-e7bb-44f3-8d96-6cb0eb9cabbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:51.712934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e269472-e7bb-44f3-8d96-6cb0eb9cabbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:51.713516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e269472-e7bb-44f3-8d96-6cb0eb9cabbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:51.715179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e269472-e7bb-44f3-8d96-6cb0eb9cabbd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:51.720905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:51.730067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1000/2019] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:51.731448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564145cf-ecc7-46bd-a816-b64568137506 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:51.741691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564145cf-ecc7-46bd-a816-b64568137506 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:51.746342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e9a9d-99d1-4aa4-8d73-591e60a9b0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:51.747041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e9a9d-99d1-4aa4-8d73-591e60a9b0f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:51.747326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e9a9d-99d1-4aa4-8d73-591e60a9b0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:51.747602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e9a9d-99d1-4aa4-8d73-591e60a9b0f6 admin admin] PATH_INFO: `/v3/projects/66f92f8d00344fbc988d7ddc2462d59f/users/59868da97897438ba5ea3b6860781bb4/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:51.763889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e9a9d-99d1-4aa4-8d73-591e60a9b0f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f92f8d00344fbc988d7ddc2462d59f, user_id=59868da97897438ba5ea3b6860781bb4, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:51.767368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e9a9d-99d1-4aa4-8d73-591e60a9b0f6 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:51.782062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1020/2020] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:51 2022] PUT /identity/v3/projects/66f92f8d00344fbc988d7ddc2462d59f/users/59868da97897438ba5ea3b6860781bb4/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:51.794271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cc454-e616-49e8-ac53-d3206d211cdd None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:51.794565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cc454-e616-49e8-ac53-d3206d211cdd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:51.794878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cc454-e616-49e8-ac53-d3206d211cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:51.829780 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-474cc454-e616-49e8-ac53-d3206d211cdd None None] MFA Rules not processed for user `59868da97897438ba5ea3b6860781bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:51.831104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-474cc454-e616-49e8-ac53-d3206d211cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:51.896940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1001/2021] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:51.908741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e9a9d-99d1-4aa4-8d73-591e60a9b0f6 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:51.910540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ad7e9a9d-99d1-4aa4-8d73-591e60a9b0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:51.971570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e9a9d-99d1-4aa4-8d73-591e60a9b0f6 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:51.975394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8e182-c119-4d4f-8371-97d2816a1f6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:51.976024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8e182-c119-4d4f-8371-97d2816a1f6d service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:51.976307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8e182-c119-4d4f-8371-97d2816a1f6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:51.976581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8e182-c119-4d4f-8371-97d2816a1f6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:51.985740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8e182-c119-4d4f-8371-97d2816a1f6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:51.987256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8e182-c119-4d4f-8371-97d2816a1f6d service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:51.996429 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1021/2022] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:52.016430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cc454-e616-49e8-ac53-d3206d211cdd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:52.079590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cc454-e616-49e8-ac53-d3206d211cdd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:52.084669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:52.085439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:52.085725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:52.086110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] PATH_INFO: `/v3/users/59868da97897438ba5ea3b6860781bb4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:52.087454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=59868da97897438ba5ea3b6860781bb4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:52.089510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:52.137885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59868da97897438ba5ea3b6860781bb4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:52.138596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59868da97897438ba5ea3b6860781bb4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:52.146724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59868da97897438ba5ea3b6860781bb4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:52.159542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59868da97897438ba5ea3b6860781bb4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:52.166527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59868da97897438ba5ea3b6860781bb4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:52.172445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1002/2023] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:00:52 2022] DELETE /identity/v3/users/59868da97897438ba5ea3b6860781bb4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:00:52.176898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544dd1d-8d5c-4531-afb0-d83987b54bae None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:52.177186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544dd1d-8d5c-4531-afb0-d83987b54bae None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:52.177494 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544dd1d-8d5c-4531-afb0-d83987b54bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:52.195941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f544dd1d-8d5c-4531-afb0-d83987b54bae None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:52.197170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f544dd1d-8d5c-4531-afb0-d83987b54bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:52.243694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:52.258153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1022/2024] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:52.268493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:52.269919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:52.281261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:52.330079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae02d3-3ace-4b75-9642-e1c4e59460bc admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:52.333988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec6f69-72fd-4f72-aeff-7fa42f945fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:52.334608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec6f69-72fd-4f72-aeff-7fa42f945fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:52.334925 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec6f69-72fd-4f72-aeff-7fa42f945fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:52.335199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec6f69-72fd-4f72-aeff-7fa42f945fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:52.345470 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec6f69-72fd-4f72-aeff-7fa42f945fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:52.347059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec6f69-72fd-4f72-aeff-7fa42f945fd2 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:52.357677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1003/2025] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:00:52.640244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544dd1d-8d5c-4531-afb0-d83987b54bae None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:52.641665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f544dd1d-8d5c-4531-afb0-d83987b54bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:52.660098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544dd1d-8d5c-4531-afb0-d83987b54bae None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:52.665281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:52.665985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:52.666196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:52.666474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] PATH_INFO: `/v3/projects/66f92f8d00344fbc988d7ddc2462d59f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:52.667827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f92f8d00344fbc988d7ddc2462d59f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:52.669749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:52.717330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f92f8d00344fbc988d7ddc2462d59f', 'request_id': 'req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:52.723261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f92f8d00344fbc988d7ddc2462d59f', 'request_id': 'req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:52.729396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f92f8d00344fbc988d7ddc2462d59f', 'request_id': 'req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:52.741782 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] The token cache is being invalidate because project 66f92f8d00344fbc988d7ddc2462d59f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:00:52.742070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:52.742681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0ebc96ee-14bf-4f1f-9b4a-e020eea9407e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:00:52.744205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1023/2026] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:00:52 2022] DELETE /identity/v3/projects/66f92f8d00344fbc988d7ddc2462d59f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:52.749341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c668ed0-cb5b-46a8-ae10-e49211ef18aa None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:52.749805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c668ed0-cb5b-46a8-ae10-e49211ef18aa None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:52.749895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c668ed0-cb5b-46a8-ae10-e49211ef18aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:52.769843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0c668ed0-cb5b-46a8-ae10-e49211ef18aa None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:52.771084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0c668ed0-cb5b-46a8-ae10-e49211ef18aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:52.836735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1004/2027] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:52.838750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce4d45-f5a5-4139-8c13-de293dc4c301 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:52.839065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce4d45-f5a5-4139-8c13-de293dc4c301 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:52.839336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce4d45-f5a5-4139-8c13-de293dc4c301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:52.860199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c7ce4d45-f5a5-4139-8c13-de293dc4c301 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:52.861398 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c7ce4d45-f5a5-4139-8c13-de293dc4c301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:52.869592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1024/2028] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:52.873903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c668ed0-cb5b-46a8-ae10-e49211ef18aa None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:52.883944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c668ed0-cb5b-46a8-ae10-e49211ef18aa None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:52.888582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b3771-9b82-4e17-a423-5638b58614f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:52.889230 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b3771-9b82-4e17-a423-5638b58614f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:52.890023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b3771-9b82-4e17-a423-5638b58614f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:52.890297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b3771-9b82-4e17-a423-5638b58614f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:52.891030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b3771-9b82-4e17-a423-5638b58614f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:00:52.891337 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b3771-9b82-4e17-a423-5638b58614f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:52.892915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b3771-9b82-4e17-a423-5638b58614f8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:52.904911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1005/2029] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:00:52.910110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce4d45-f5a5-4139-8c13-de293dc4c301 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:52.920432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce4d45-f5a5-4139-8c13-de293dc4c301 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:52.925393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4006ca-f438-45c7-b30f-1cad3b65e799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:52.926047 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4006ca-f438-45c7-b30f-1cad3b65e799 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:52.926307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4006ca-f438-45c7-b30f-1cad3b65e799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:52.926585 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4006ca-f438-45c7-b30f-1cad3b65e799 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:52.927283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4006ca-f438-45c7-b30f-1cad3b65e799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:52.929203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4006ca-f438-45c7-b30f-1cad3b65e799 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:52.958817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1025/2030] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:00:52 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:00:52.962478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b3771-9b82-4e17-a423-5638b58614f8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:52.972952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b3771-9b82-4e17-a423-5638b58614f8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:53.033336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bf86d-2723-4743-96d6-a4708d298ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:53.033976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bf86d-2723-4743-96d6-a4708d298ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:53.034265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bf86d-2723-4743-96d6-a4708d298ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:53.034545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bf86d-2723-4743-96d6-a4708d298ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:53.035270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bf86d-2723-4743-96d6-a4708d298ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:53.037201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bf86d-2723-4743-96d6-a4708d298ed1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:53.062516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1006/2031] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:00:52 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:00:53.066449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4006ca-f438-45c7-b30f-1cad3b65e799 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:53.078166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4006ca-f438-45c7-b30f-1cad3b65e799 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:53.083212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c761b-513e-4af2-908e-b6ec282df94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:53.083917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c761b-513e-4af2-908e-b6ec282df94c admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:53.084267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c761b-513e-4af2-908e-b6ec282df94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:53.084617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c761b-513e-4af2-908e-b6ec282df94c admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:53.085518 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c761b-513e-4af2-908e-b6ec282df94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:53.087256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c761b-513e-4af2-908e-b6ec282df94c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:53.097794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1026/2032] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:53.101967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bf86d-2723-4743-96d6-a4708d298ed1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:53.112743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bf86d-2723-4743-96d6-a4708d298ed1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:53.118823 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f7035-1b82-40ba-b559-4a992bcbf807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:53.119648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f7035-1b82-40ba-b559-4a992bcbf807 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:53.120085 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f7035-1b82-40ba-b559-4a992bcbf807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:53.120656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f7035-1b82-40ba-b559-4a992bcbf807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:53.121419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f7035-1b82-40ba-b559-4a992bcbf807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:53.123167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f7035-1b82-40ba-b559-4a992bcbf807 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:53.136132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1007/2033] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:00:53.140543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c761b-513e-4af2-908e-b6ec282df94c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:53.154716 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c761b-513e-4af2-908e-b6ec282df94c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:53.161535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51c793-b1b4-4dff-8734-37c25452b697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:53.161535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51c793-b1b4-4dff-8734-37c25452b697 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:53.161535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51c793-b1b4-4dff-8734-37c25452b697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:53.161987 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51c793-b1b4-4dff-8734-37c25452b697 admin admin] PATH_INFO: `/v3/projects/cacd493eb4a74353a870292eec25d6fc/users/0d71850d263d4a458463c50168690640/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:53.194898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51c793-b1b4-4dff-8734-37c25452b697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cacd493eb4a74353a870292eec25d6fc, user_id=0d71850d263d4a458463c50168690640, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:53.197741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51c793-b1b4-4dff-8734-37c25452b697 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:53.213938 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1027/2034] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:00:53 2022] PUT /identity/v3/projects/cacd493eb4a74353a870292eec25d6fc/users/0d71850d263d4a458463c50168690640/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:00:53.224477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e83c3-4d74-4c6b-a5b0-c82109a5316c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:53.224771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e83c3-4d74-4c6b-a5b0-c82109a5316c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:53.225051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e83c3-4d74-4c6b-a5b0-c82109a5316c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:53.256212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d47e83c3-4d74-4c6b-a5b0-c82109a5316c None None] MFA Rules not processed for user `0d71850d263d4a458463c50168690640`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:00:53.258224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d47e83c3-4d74-4c6b-a5b0-c82109a5316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:53.267508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:00:53.359755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1008/2035] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:00:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:00:53.370933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51c793-b1b4-4dff-8734-37c25452b697 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:00:53.374771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5c51c793-b1b4-4dff-8734-37c25452b697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:00:53.442711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51c793-b1b4-4dff-8734-37c25452b697 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:00:53.446750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0956f-683e-4c29-a7f1-e7411255eb65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:00:53.447694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0956f-683e-4c29-a7f1-e7411255eb65 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:00:53.448075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0956f-683e-4c29-a7f1-e7411255eb65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:00:53.448367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0956f-683e-4c29-a7f1-e7411255eb65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:00:53.459875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0956f-683e-4c29-a7f1-e7411255eb65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:00:53.461345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0956f-683e-4c29-a7f1-e7411255eb65 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:00:53.472711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1028/2036] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:00:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:03.353576 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=24 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:03.468227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:04.029275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e83c3-4d74-4c6b-a5b0-c82109a5316c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:04.106418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e83c3-4d74-4c6b-a5b0-c82109a5316c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:04.114342 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:04.114342 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:04.114342 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:04.114342 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] PATH_INFO: `/v3/users/0d71850d263d4a458463c50168690640` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:04.114688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d71850d263d4a458463c50168690640)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:04.116415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:04.172928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d71850d263d4a458463c50168690640'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:04.173692 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d71850d263d4a458463c50168690640'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:04.182737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d71850d263d4a458463c50168690640'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:04.202290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d71850d263d4a458463c50168690640'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:04.210993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d71850d263d4a458463c50168690640'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:04.225386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1009/2037] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:04 2022] DELETE /identity/v3/users/0d71850d263d4a458463c50168690640 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:04.226738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2583922-d97d-4f34-853d-5e0ac96c384e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:04.227061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2583922-d97d-4f34-853d-5e0ac96c384e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:04.227345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2583922-d97d-4f34-853d-5e0ac96c384e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:04.275948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d2583922-d97d-4f34-853d-5e0ac96c384e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:04.277193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d2583922-d97d-4f34-853d-5e0ac96c384e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:04.353989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1029/2038] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:04.367051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:04.368724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:04.438249 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195c35f-975c-4d5c-be0f-bee16595ca05 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:04.442748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a6b38-c259-414b-a4f6-8adc47460405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:04.443423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a6b38-c259-414b-a4f6-8adc47460405 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:04.443706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a6b38-c259-414b-a4f6-8adc47460405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:04.443980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a6b38-c259-414b-a4f6-8adc47460405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:04.454843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a6b38-c259-414b-a4f6-8adc47460405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:04.456206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a6b38-c259-414b-a4f6-8adc47460405 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:04.469890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1010/2039] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:04.884173 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2583922-d97d-4f34-853d-5e0ac96c384e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:04.885698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d2583922-d97d-4f34-853d-5e0ac96c384e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:04.906302 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2583922-d97d-4f34-853d-5e0ac96c384e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:04.911400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:04.912042 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:04.912331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:04.912610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] PATH_INFO: `/v3/projects/cacd493eb4a74353a870292eec25d6fc` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:04.915157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cacd493eb4a74353a870292eec25d6fc)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:04.917325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:05.005817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cacd493eb4a74353a870292eec25d6fc', 'request_id': 'req-9fcb93a0-67c1-47bc-afef-a40b80a60e01'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:05.017074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cacd493eb4a74353a870292eec25d6fc', 'request_id': 'req-9fcb93a0-67c1-47bc-afef-a40b80a60e01'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:05.039715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cacd493eb4a74353a870292eec25d6fc', 'request_id': 'req-9fcb93a0-67c1-47bc-afef-a40b80a60e01'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:05.053337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] The token cache is being invalidate because project cacd493eb4a74353a870292eec25d6fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:05.053645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:05.054189 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9fcb93a0-67c1-47bc-afef-a40b80a60e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:05.055853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1030/2040] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:04 2022] DELETE /identity/v3/projects/cacd493eb4a74353a870292eec25d6fc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:05.061783 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7472b0-3a7f-4b88-a624-420c2aaa74f0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:05.062202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7472b0-3a7f-4b88-a624-420c2aaa74f0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:05.062486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7472b0-3a7f-4b88-a624-420c2aaa74f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:05.083570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bd7472b0-3a7f-4b88-a624-420c2aaa74f0 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:05.084996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bd7472b0-3a7f-4b88-a624-420c2aaa74f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:05.108887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:05.172247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1011/2041] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:05.180104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d1450-3654-43a9-9827-4552e4a07039 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:05.180395 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d1450-3654-43a9-9827-4552e4a07039 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:05.180680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d1450-3654-43a9-9827-4552e4a07039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:05.203634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f04d1450-3654-43a9-9827-4552e4a07039 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:05.205009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f04d1450-3654-43a9-9827-4552e4a07039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:05.214819 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1031/2042] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:05.220448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7472b0-3a7f-4b88-a624-420c2aaa74f0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:05.232504 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7472b0-3a7f-4b88-a624-420c2aaa74f0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:05.238950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae0bc-7c39-4a0e-9903-fcb88515e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:05.239177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae0bc-7c39-4a0e-9903-fcb88515e206 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:05.239668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae0bc-7c39-4a0e-9903-fcb88515e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:05.239929 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae0bc-7c39-4a0e-9903-fcb88515e206 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:05.241274 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ae0bc-7c39-4a0e-9903-fcb88515e206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:05.241978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ae0bc-7c39-4a0e-9903-fcb88515e206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:05.243373 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ae0bc-7c39-4a0e-9903-fcb88515e206 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:05.257218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1012/2043] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:01:05.261725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d1450-3654-43a9-9827-4552e4a07039 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:05.272508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d1450-3654-43a9-9827-4552e4a07039 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:05.278081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d543511-dd64-4134-9b39-4909a1731331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:05.278906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d543511-dd64-4134-9b39-4909a1731331 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:05.280184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d543511-dd64-4134-9b39-4909a1731331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:05.280439 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d543511-dd64-4134-9b39-4909a1731331 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:05.281592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d543511-dd64-4134-9b39-4909a1731331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:05.283828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d543511-dd64-4134-9b39-4909a1731331 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:05.331738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1032/2044] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:05 2022] POST /identity/v3/projects => generated 359 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:05.339829 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae0bc-7c39-4a0e-9903-fcb88515e206 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:05.354332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae0bc-7c39-4a0e-9903-fcb88515e206 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:05.486667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c22106-f59b-4876-bdeb-6891b97f378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:05.488346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22106-f59b-4876-bdeb-6891b97f378a admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:05.488821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22106-f59b-4876-bdeb-6891b97f378a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:05.489281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22106-f59b-4876-bdeb-6891b97f378a admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:05.490300 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c22106-f59b-4876-bdeb-6891b97f378a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:05.493320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c22106-f59b-4876-bdeb-6891b97f378a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:05.525760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1013/2045] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:05 2022] POST /identity/v3/users => generated 280 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:05.529470 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d543511-dd64-4134-9b39-4909a1731331 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:05.562224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d543511-dd64-4134-9b39-4909a1731331 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:05.564991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee39f0c-040f-4986-a1bc-b7c8ecc2192a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:05.565629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee39f0c-040f-4986-a1bc-b7c8ecc2192a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:05.565913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee39f0c-040f-4986-a1bc-b7c8ecc2192a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:05.566966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee39f0c-040f-4986-a1bc-b7c8ecc2192a admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:05.568105 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee39f0c-040f-4986-a1bc-b7c8ecc2192a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:05.570164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee39f0c-040f-4986-a1bc-b7c8ecc2192a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:05.594174 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1033/2046] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:01:05.595777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c22106-f59b-4876-bdeb-6891b97f378a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:05.606259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c22106-f59b-4876-bdeb-6891b97f378a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:05.622296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59b2e5-2907-4918-87de-eba4b19e572c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:05.622972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59b2e5-2907-4918-87de-eba4b19e572c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:05.623255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59b2e5-2907-4918-87de-eba4b19e572c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:05.624166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59b2e5-2907-4918-87de-eba4b19e572c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:05.624874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59b2e5-2907-4918-87de-eba4b19e572c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:05.626488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59b2e5-2907-4918-87de-eba4b19e572c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:05.644992 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1014/2047] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:01:05.646360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee39f0c-040f-4986-a1bc-b7c8ecc2192a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:05.668236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee39f0c-040f-4986-a1bc-b7c8ecc2192a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:05.684523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b78e7-74de-46bb-8aef-49d7cf6058f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:05.685536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b78e7-74de-46bb-8aef-49d7cf6058f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:05.686120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b78e7-74de-46bb-8aef-49d7cf6058f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:05.686633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b78e7-74de-46bb-8aef-49d7cf6058f6 admin admin] PATH_INFO: `/v3/projects/af51b56bec0c46dc9c5635a2d65a8ccc/users/3d6da50e2ba9483a955299f97780f148/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:05.723825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b78e7-74de-46bb-8aef-49d7cf6058f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af51b56bec0c46dc9c5635a2d65a8ccc, user_id=3d6da50e2ba9483a955299f97780f148, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:05.728181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b78e7-74de-46bb-8aef-49d7cf6058f6 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:05.751451 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1034/2048] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:05 2022] PUT /identity/v3/projects/af51b56bec0c46dc9c5635a2d65a8ccc/users/3d6da50e2ba9483a955299f97780f148/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:05.761652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359ab8-729a-40da-aef8-4ab726c0bf19 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:05.761982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359ab8-729a-40da-aef8-4ab726c0bf19 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:05.762207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359ab8-729a-40da-aef8-4ab726c0bf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:05.817251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-68359ab8-729a-40da-aef8-4ab726c0bf19 None None] MFA Rules not processed for user `3d6da50e2ba9483a955299f97780f148`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:05.822088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-68359ab8-729a-40da-aef8-4ab726c0bf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:05.966957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1015/2049] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:05 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:06.007342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b78e7-74de-46bb-8aef-49d7cf6058f6 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:06.009530 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-780b78e7-74de-46bb-8aef-49d7cf6058f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:06.036518 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:06.076828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b78e7-74de-46bb-8aef-49d7cf6058f6 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:06.080988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d337a33-c23a-4a93-81fe-c17d16080a3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:06.081690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d337a33-c23a-4a93-81fe-c17d16080a3a service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:06.082044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d337a33-c23a-4a93-81fe-c17d16080a3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:06.082421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d337a33-c23a-4a93-81fe-c17d16080a3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:06.095046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d337a33-c23a-4a93-81fe-c17d16080a3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:06.096602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d337a33-c23a-4a93-81fe-c17d16080a3a service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:06.106843 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1035/2050] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:06 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:06.125247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68359ab8-729a-40da-aef8-4ab726c0bf19 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:06.145535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:06.206978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68359ab8-729a-40da-aef8-4ab726c0bf19 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:06.211599 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:06.212281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:06.212567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:06.212848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] PATH_INFO: `/v3/users/3d6da50e2ba9483a955299f97780f148` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:06.214063 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d6da50e2ba9483a955299f97780f148)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:06.216013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:06.267431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d6da50e2ba9483a955299f97780f148'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:06.268104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d6da50e2ba9483a955299f97780f148'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:06.276327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d6da50e2ba9483a955299f97780f148'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:06.290417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d6da50e2ba9483a955299f97780f148'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:06.300225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d6da50e2ba9483a955299f97780f148'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:06.309869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1016/2051] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:06 2022] DELETE /identity/v3/users/3d6da50e2ba9483a955299f97780f148 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:06.315073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e7f3f-2c4e-4e6b-b888-2a15ec5edae1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:06.315689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e7f3f-2c4e-4e6b-b888-2a15ec5edae1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:06.316358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e7f3f-2c4e-4e6b-b888-2a15ec5edae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:06.338815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e34e7f3f-2c4e-4e6b-b888-2a15ec5edae1 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:06.340661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e34e7f3f-2c4e-4e6b-b888-2a15ec5edae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:06.403571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1036/2052] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:06.414106 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:06.415552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:06.475141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b648c1-edd2-4253-827d-5079e0680269 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:06.478682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37cc68-cc38-4d76-a709-96afebed8657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:06.479346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37cc68-cc38-4d76-a709-96afebed8657 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:06.479627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37cc68-cc38-4d76-a709-96afebed8657 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:06.479902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37cc68-cc38-4d76-a709-96afebed8657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:06.490452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37cc68-cc38-4d76-a709-96afebed8657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:06.492029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37cc68-cc38-4d76-a709-96afebed8657 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:06.505667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1017/2053] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:01:06.580955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e7f3f-2c4e-4e6b-b888-2a15ec5edae1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:06.584110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e34e7f3f-2c4e-4e6b-b888-2a15ec5edae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:06.609983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e7f3f-2c4e-4e6b-b888-2a15ec5edae1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:06.615448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:06.616116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:06.616525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:06.616809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] PATH_INFO: `/v3/users/4bdd68b67d154cfcb02982799a47b2cc` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:06.618057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bdd68b67d154cfcb02982799a47b2cc)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:06.620195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:06.696786 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bdd68b67d154cfcb02982799a47b2cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:06.697467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bdd68b67d154cfcb02982799a47b2cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:06.716353 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bdd68b67d154cfcb02982799a47b2cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:06.725296 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bdd68b67d154cfcb02982799a47b2cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:06.734901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bdd68b67d154cfcb02982799a47b2cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:06.746328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1037/2054] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:06 2022] DELETE /identity/v3/users/4bdd68b67d154cfcb02982799a47b2cc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:06.751576 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45333-5564-4a22-8dea-d1071b8c6096 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:06.751896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45333-5564-4a22-8dea-d1071b8c6096 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:06.752202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45333-5564-4a22-8dea-d1071b8c6096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:06.776390 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0dc45333-5564-4a22-8dea-d1071b8c6096 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:06.778102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0dc45333-5564-4a22-8dea-d1071b8c6096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:06.827798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:06.844087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:06.846480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:06.867834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1018/2055] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:06.888394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45333-5564-4a22-8dea-d1071b8c6096 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:06.891045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0dc45333-5564-4a22-8dea-d1071b8c6096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:06.929211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:06.978039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f355f8f-a467-4070-9914-b87e7e2675ab admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:06.988261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:06.994062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:06.994062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:06.994062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] PATH_INFO: `/v3/projects/af51b56bec0c46dc9c5635a2d65a8ccc` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:06.994685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] RBAC: Authorizing `identity:delete_project(project_id=af51b56bec0c46dc9c5635a2d65a8ccc)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:06.996745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:06.997745 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45333-5564-4a22-8dea-d1071b8c6096 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:07.004544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80728b-063d-49ad-9e0f-b8cc38300ef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:07.004909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80728b-063d-49ad-9e0f-b8cc38300ef0 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:07.005192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80728b-063d-49ad-9e0f-b8cc38300ef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:07.005465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80728b-063d-49ad-9e0f-b8cc38300ef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:07.033082 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80728b-063d-49ad-9e0f-b8cc38300ef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:07.035482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80728b-063d-49ad-9e0f-b8cc38300ef0 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:07.061158 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1019/2056] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:01:07.081236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af51b56bec0c46dc9c5635a2d65a8ccc', 'request_id': 'req-3caec8fa-c1f6-45ce-b14e-028a6015504b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:07.091897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af51b56bec0c46dc9c5635a2d65a8ccc', 'request_id': 'req-3caec8fa-c1f6-45ce-b14e-028a6015504b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:07.102387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af51b56bec0c46dc9c5635a2d65a8ccc', 'request_id': 'req-3caec8fa-c1f6-45ce-b14e-028a6015504b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:07.119085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] The token cache is being invalidate because project af51b56bec0c46dc9c5635a2d65a8ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:07.119421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:07.120121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:07.122375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1038/2057] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:06 2022] DELETE /identity/v3/projects/af51b56bec0c46dc9c5635a2d65a8ccc => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:07.671724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80728b-063d-49ad-9e0f-b8cc38300ef0 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:07.673691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5f80728b-063d-49ad-9e0f-b8cc38300ef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:07.688939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80728b-063d-49ad-9e0f-b8cc38300ef0 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:07.759131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:07.760486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:07.760808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:07.761364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] PATH_INFO: `/v3/projects/551ad8000fca49e181e661b108a43a8e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:07.763141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=551ad8000fca49e181e661b108a43a8e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:07.767110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:07.823819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '551ad8000fca49e181e661b108a43a8e', 'request_id': 'req-19ce1337-c729-4332-9087-5f9721326d56'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:07.831734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '551ad8000fca49e181e661b108a43a8e', 'request_id': 'req-19ce1337-c729-4332-9087-5f9721326d56'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:07.842640 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '551ad8000fca49e181e661b108a43a8e', 'request_id': 'req-19ce1337-c729-4332-9087-5f9721326d56'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:07.858561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] The token cache is being invalidate because project 551ad8000fca49e181e661b108a43a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:07.858916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:07.859614 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:07.864071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1020/2058] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:07 2022] DELETE /identity/v3/projects/551ad8000fca49e181e661b108a43a8e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:07.868045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:07.869775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:07.884908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caec8fa-c1f6-45ce-b14e-028a6015504b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:07.961737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:07.962392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:07.962683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:07.962989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] PATH_INFO: `/v3/users/46eaf8b9c11c41d0bbe86e92bccf3486` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:07.964245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=46eaf8b9c11c41d0bbe86e92bccf3486)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:07.966232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:07.991423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:07.992957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:08.007651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce1337-c729-4332-9087-5f9721326d56 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:08.012364 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:08.013003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:08.013282 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:08.013554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] PATH_INFO: `/v3/users/d09411d75019428a922e12e02c72f4af` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:08.015355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d09411d75019428a922e12e02c72f4af)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:08.017320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:08.022749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46eaf8b9c11c41d0bbe86e92bccf3486'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.023440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46eaf8b9c11c41d0bbe86e92bccf3486'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.032072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:08.037710 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46eaf8b9c11c41d0bbe86e92bccf3486'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.042247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:08.049651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46eaf8b9c11c41d0bbe86e92bccf3486'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.058906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46eaf8b9c11c41d0bbe86e92bccf3486'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.067179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1039/2059] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:07 2022] DELETE /identity/v3/users/46eaf8b9c11c41d0bbe86e92bccf3486 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:08.080513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd09411d75019428a922e12e02c72f4af'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.081197 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd09411d75019428a922e12e02c72f4af'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.088962 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd09411d75019428a922e12e02c72f4af'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.102737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd09411d75019428a922e12e02c72f4af'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.111113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd09411d75019428a922e12e02c72f4af'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.117295 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1021/2060] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:07 2022] DELETE /identity/v3/users/d09411d75019428a922e12e02c72f4af => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:01:08.392331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:08.394758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:08.450573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:08.456659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:08.515894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb045b5-5ff5-4a99-8805-3c730912e9aa admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:08.574098 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f579e-a6ef-4e50-98c5-c2386cc67f2f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:08.581193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:08.582648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:08.582973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:08.583248 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] PATH_INFO: `/v3/projects/f0cd5be4a02f41eaa8a6d3995991d2bf` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:08.584445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0cd5be4a02f41eaa8a6d3995991d2bf)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:08.586445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:08.618249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:08.619089 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:08.619480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:08.619861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] PATH_INFO: `/v3/projects/cb2697d81b164ddfbb33acd843f461f6` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:08.621164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb2697d81b164ddfbb33acd843f461f6)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:08.623398 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:08.654136 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0cd5be4a02f41eaa8a6d3995991d2bf', 'request_id': 'req-2dd6a977-5566-4ccf-b7c1-700877b13a36'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.663828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0cd5be4a02f41eaa8a6d3995991d2bf', 'request_id': 'req-2dd6a977-5566-4ccf-b7c1-700877b13a36'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.675055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0cd5be4a02f41eaa8a6d3995991d2bf', 'request_id': 'req-2dd6a977-5566-4ccf-b7c1-700877b13a36'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.683160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb2697d81b164ddfbb33acd843f461f6', 'request_id': 'req-0cc630a2-2695-481e-a292-1858161eda1e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.688171 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] The token cache is being invalidate because project f0cd5be4a02f41eaa8a6d3995991d2bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:08.688467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.689496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2dd6a977-5566-4ccf-b7c1-700877b13a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.690247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb2697d81b164ddfbb33acd843f461f6', 'request_id': 'req-0cc630a2-2695-481e-a292-1858161eda1e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.691213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1022/2061] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:08 2022] DELETE /identity/v3/projects/f0cd5be4a02f41eaa8a6d3995991d2bf => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:08.696104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c6590-273e-42af-9416-7b375e114966 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:08.696395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c6590-273e-42af-9416-7b375e114966 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:08.696674 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c6590-273e-42af-9416-7b375e114966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:08.698200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb2697d81b164ddfbb33acd843f461f6', 'request_id': 'req-0cc630a2-2695-481e-a292-1858161eda1e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.713171 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] The token cache is being invalidate because project cb2697d81b164ddfbb33acd843f461f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:08.713455 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.714149 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0cc630a2-2695-481e-a292-1858161eda1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:08.715711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1040/2062] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:08 2022] DELETE /identity/v3/projects/cb2697d81b164ddfbb33acd843f461f6 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:08.720200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6b2c6590-273e-42af-9416-7b375e114966 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:08.721007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3a12e-2a35-4147-9054-d4152a887255 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:08.721299 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3a12e-2a35-4147-9054-d4152a887255 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:08.721587 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3a12e-2a35-4147-9054-d4152a887255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:08.721700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6b2c6590-273e-42af-9416-7b375e114966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:08.733499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:08.744322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-93f3a12e-2a35-4147-9054-d4152a887255 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:08.746055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-93f3a12e-2a35-4147-9054-d4152a887255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:08.767602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:08.804303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1023/2063] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:08.812325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ca1c7-3da2-4d8b-b654-e4ccf884574b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:08.812606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ca1c7-3da2-4d8b-b654-e4ccf884574b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:08.812931 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ca1c7-3da2-4d8b-b654-e4ccf884574b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:08.824902 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1041/2064] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:08.833153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b23ca1c7-3da2-4d8b-b654-e4ccf884574b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:08.834130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e6830-4c5a-47f0-ad0f-dcb2197e4194 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:08.834272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b23ca1c7-3da2-4d8b-b654-e4ccf884574b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:08.834426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e6830-4c5a-47f0-ad0f-dcb2197e4194 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:08.834696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e6830-4c5a-47f0-ad0f-dcb2197e4194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:08.846066 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1024/2065] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:08.847502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ca1c7-3da2-4d8b-b654-e4ccf884574b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:08.861214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ca9e6830-4c5a-47f0-ad0f-dcb2197e4194 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:08.862285 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ca1c7-3da2-4d8b-b654-e4ccf884574b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:08.862555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ca9e6830-4c5a-47f0-ad0f-dcb2197e4194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:08.868481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be4506-d2e5-4c77-b566-09130f9bfdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:08.869213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be4506-d2e5-4c77-b566-09130f9bfdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:08.869504 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be4506-d2e5-4c77-b566-09130f9bfdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:08.869787 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be4506-d2e5-4c77-b566-09130f9bfdf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:08.870526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be4506-d2e5-4c77-b566-09130f9bfdf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:08.870810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be4506-d2e5-4c77-b566-09130f9bfdf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:08.872426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be4506-d2e5-4c77-b566-09130f9bfdf6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:08.877272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1042/2066] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:08.878974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e6830-4c5a-47f0-ad0f-dcb2197e4194 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:08.885888 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1025/2067] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:01:08.890810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be4506-d2e5-4c77-b566-09130f9bfdf6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:08.891444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e6830-4c5a-47f0-ad0f-dcb2197e4194 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:08.897575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450505f-37a6-4d96-b1a7-917ba4110411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:08.898882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450505f-37a6-4d96-b1a7-917ba4110411 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:08.899169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450505f-37a6-4d96-b1a7-917ba4110411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:08.899447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450505f-37a6-4d96-b1a7-917ba4110411 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:08.900215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450505f-37a6-4d96-b1a7-917ba4110411 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:08.900528 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450505f-37a6-4d96-b1a7-917ba4110411 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:08.902197 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450505f-37a6-4d96-b1a7-917ba4110411 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:08.902733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be4506-d2e5-4c77-b566-09130f9bfdf6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:08.907508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919c4a5-a3e9-41d3-b336-8a877643f6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:08.908118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919c4a5-a3e9-41d3-b336-8a877643f6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:08.908409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919c4a5-a3e9-41d3-b336-8a877643f6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:08.908687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919c4a5-a3e9-41d3-b336-8a877643f6e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:08.909355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919c4a5-a3e9-41d3-b336-8a877643f6e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:08.911385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919c4a5-a3e9-41d3-b336-8a877643f6e3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:08.914396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1043/2068] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:01:08.918671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450505f-37a6-4d96-b1a7-917ba4110411 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:08.928357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450505f-37a6-4d96-b1a7-917ba4110411 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:08.933341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea77fb-9925-411b-91df-918599929839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:08.934139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea77fb-9925-411b-91df-918599929839 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:08.934489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea77fb-9925-411b-91df-918599929839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:08.934886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea77fb-9925-411b-91df-918599929839 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:08.935618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea77fb-9925-411b-91df-918599929839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:08.938220 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea77fb-9925-411b-91df-918599929839 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:08.943431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1026/2069] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:08 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:08.944726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919c4a5-a3e9-41d3-b336-8a877643f6e3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:08.970139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1044/2070] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:08 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:08.978810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea77fb-9925-411b-91df-918599929839 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.024693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919c4a5-a3e9-41d3-b336-8a877643f6e3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.057169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea77fb-9925-411b-91df-918599929839 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.064535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77b5ae-9cfa-4982-9028-37cbb4bd97ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.065250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77b5ae-9cfa-4982-9028-37cbb4bd97ec admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.065887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77b5ae-9cfa-4982-9028-37cbb4bd97ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.066208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77b5ae-9cfa-4982-9028-37cbb4bd97ec admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.066936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77b5ae-9cfa-4982-9028-37cbb4bd97ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.069023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77b5ae-9cfa-4982-9028-37cbb4bd97ec admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.094349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1045/2071] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:08 2022] POST /identity/v3/users => generated 282 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:09.095605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7b79a-88b0-4a4c-bc15-a5ec237fa295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.096440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7b79a-88b0-4a4c-bc15-a5ec237fa295 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.096925 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7b79a-88b0-4a4c-bc15-a5ec237fa295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.097280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7b79a-88b0-4a4c-bc15-a5ec237fa295 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.097952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7b79a-88b0-4a4c-bc15-a5ec237fa295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.098746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77b5ae-9cfa-4982-9028-37cbb4bd97ec admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.100101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7b79a-88b0-4a4c-bc15-a5ec237fa295 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.109440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77b5ae-9cfa-4982-9028-37cbb4bd97ec admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.116588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fd45b-d0bd-4454-9059-46f1d770ffc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.117743 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fd45b-d0bd-4454-9059-46f1d770ffc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.118354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fd45b-d0bd-4454-9059-46f1d770ffc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.118963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fd45b-d0bd-4454-9059-46f1d770ffc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.120341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699fd45b-d0bd-4454-9059-46f1d770ffc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.124104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699fd45b-d0bd-4454-9059-46f1d770ffc2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.127772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1027/2072] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:08 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:09.131460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7b79a-88b0-4a4c-bc15-a5ec237fa295 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.138582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1046/2073] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:01:09.143176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fd45b-d0bd-4454-9059-46f1d770ffc2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.144072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7b79a-88b0-4a4c-bc15-a5ec237fa295 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.149235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48cdb6-c027-4b31-b6dd-470bc0aa2b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.149999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48cdb6-c027-4b31-b6dd-470bc0aa2b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.150420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48cdb6-c027-4b31-b6dd-470bc0aa2b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.150706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48cdb6-c027-4b31-b6dd-470bc0aa2b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.151500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48cdb6-c027-4b31-b6dd-470bc0aa2b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.153102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48cdb6-c027-4b31-b6dd-470bc0aa2b37 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.155444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fd45b-d0bd-4454-9059-46f1d770ffc2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.159215 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:09.161442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648db0d1-5887-488b-b133-cb2da11c64fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.162221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648db0d1-5887-488b-b133-cb2da11c64fe admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.162506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648db0d1-5887-488b-b133-cb2da11c64fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.163060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648db0d1-5887-488b-b133-cb2da11c64fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.163772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648db0d1-5887-488b-b133-cb2da11c64fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.165364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648db0d1-5887-488b-b133-cb2da11c64fe admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.169397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1028/2074] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:01:09.171098 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48cdb6-c027-4b31-b6dd-470bc0aa2b37 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.178503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1047/2075] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:01:09.182864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648db0d1-5887-488b-b133-cb2da11c64fe admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.183799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48cdb6-c027-4b31-b6dd-470bc0aa2b37 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.189165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1eaaa-de5a-413d-811e-61670341de5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.189776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1eaaa-de5a-413d-811e-61670341de5d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.190056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1eaaa-de5a-413d-811e-61670341de5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.190329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1eaaa-de5a-413d-811e-61670341de5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.191083 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1eaaa-de5a-413d-811e-61670341de5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.192671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1eaaa-de5a-413d-811e-61670341de5d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.193909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648db0d1-5887-488b-b133-cb2da11c64fe admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.198800 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9de4dc-aebf-41dd-a415-f18c0fdcda07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.199480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9de4dc-aebf-41dd-a415-f18c0fdcda07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.199766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9de4dc-aebf-41dd-a415-f18c0fdcda07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.200045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9de4dc-aebf-41dd-a415-f18c0fdcda07 admin admin] PATH_INFO: `/v3/projects/c23618da05ef412380473903e08a2109/users/0f59f99c000c4cbeb36628a8e6c21f31/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.203969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1029/2076] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:09.207999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1eaaa-de5a-413d-811e-61670341de5d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.216257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9de4dc-aebf-41dd-a415-f18c0fdcda07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c23618da05ef412380473903e08a2109, user_id=0f59f99c000c4cbeb36628a8e6c21f31, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.217735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1eaaa-de5a-413d-811e-61670341de5d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.219484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9de4dc-aebf-41dd-a415-f18c0fdcda07 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.222705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de651e4f-e099-4a57-b9e0-a6bffbe26654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.223377 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de651e4f-e099-4a57-b9e0-a6bffbe26654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.223659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de651e4f-e099-4a57-b9e0-a6bffbe26654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.223993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de651e4f-e099-4a57-b9e0-a6bffbe26654 admin admin] PATH_INFO: `/v3/projects/043054133ca246f7a0ca0fbdf3fd6707/users/c604ee24070f4349805ba9311926f87a/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.234836 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1048/2077] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:09 2022] PUT /identity/v3/projects/c23618da05ef412380473903e08a2109/users/0f59f99c000c4cbeb36628a8e6c21f31/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:01:09.244181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de651e4f-e099-4a57-b9e0-a6bffbe26654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=043054133ca246f7a0ca0fbdf3fd6707, user_id=c604ee24070f4349805ba9311926f87a, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.245262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8786516-0473-43a7-8fed-6927d4fa59be None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.245551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8786516-0473-43a7-8fed-6927d4fa59be None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.245866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8786516-0473-43a7-8fed-6927d4fa59be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.246862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de651e4f-e099-4a57-b9e0-a6bffbe26654 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.261535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1030/2078] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:09 2022] PUT /identity/v3/projects/043054133ca246f7a0ca0fbdf3fd6707/users/c604ee24070f4349805ba9311926f87a/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:01:09.273223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e674a-68d2-4666-91da-78bcdb557ace None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.273593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e674a-68d2-4666-91da-78bcdb557ace None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.273974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e674a-68d2-4666-91da-78bcdb557ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.278725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d8786516-0473-43a7-8fed-6927d4fa59be None None] MFA Rules not processed for user `0f59f99c000c4cbeb36628a8e6c21f31`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:09.280185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d8786516-0473-43a7-8fed-6927d4fa59be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:09.296948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:09.312077 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-975e674a-68d2-4666-91da-78bcdb557ace None None] MFA Rules not processed for user `c604ee24070f4349805ba9311926f87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:09.313739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-975e674a-68d2-4666-91da-78bcdb557ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:09.365396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1049/2079] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:09 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:09.369811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8786516-0473-43a7-8fed-6927d4fa59be None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.396138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1031/2080] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:09.400598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e674a-68d2-4666-91da-78bcdb557ace None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.436712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8786516-0473-43a7-8fed-6927d4fa59be None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.441964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff5741-d4be-4a06-ac7c-bb7af2758105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.442300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff5741-d4be-4a06-ac7c-bb7af2758105 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.442578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff5741-d4be-4a06-ac7c-bb7af2758105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.442879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff5741-d4be-4a06-ac7c-bb7af2758105 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.443553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff5741-d4be-4a06-ac7c-bb7af2758105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.445473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff5741-d4be-4a06-ac7c-bb7af2758105 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.474951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1050/2081] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:09 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:09.477539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e674a-68d2-4666-91da-78bcdb557ace None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.479656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff5741-d4be-4a06-ac7c-bb7af2758105 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.546878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b55cf-3929-4069-8fd0-b901b28ab614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.548139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b55cf-3929-4069-8fd0-b901b28ab614 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.548139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b55cf-3929-4069-8fd0-b901b28ab614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.548139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b55cf-3929-4069-8fd0-b901b28ab614 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.548785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b55cf-3929-4069-8fd0-b901b28ab614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.553820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b55cf-3929-4069-8fd0-b901b28ab614 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.556987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:09.560656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff5741-d4be-4a06-ac7c-bb7af2758105 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.568761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491ba6d-0274-4921-a6e4-98d234105ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.570109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ba6d-0274-4921-a6e4-98d234105ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.570659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ba6d-0274-4921-a6e4-98d234105ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.570850 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ba6d-0274-4921-a6e4-98d234105ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.571565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491ba6d-0274-4921-a6e4-98d234105ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.573860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491ba6d-0274-4921-a6e4-98d234105ef9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.588780 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1032/2082] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:09 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:09.593407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b55cf-3929-4069-8fd0-b901b28ab614 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.601865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1051/2083] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:09 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:09.603090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491ba6d-0274-4921-a6e4-98d234105ef9 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.613969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491ba6d-0274-4921-a6e4-98d234105ef9 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.659066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:09.678206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b55cf-3929-4069-8fd0-b901b28ab614 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.685132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae666005-e0b5-420f-9c84-da2d69d66a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.686661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae666005-e0b5-420f-9c84-da2d69d66a25 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.687194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae666005-e0b5-420f-9c84-da2d69d66a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.687630 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae666005-e0b5-420f-9c84-da2d69d66a25 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.688463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae666005-e0b5-420f-9c84-da2d69d66a25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.690651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae666005-e0b5-420f-9c84-da2d69d66a25 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.704635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ff6ca-56a4-4b20-9a0c-ef564f0fbdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.706387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ff6ca-56a4-4b20-9a0c-ef564f0fbdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.706806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ff6ca-56a4-4b20-9a0c-ef564f0fbdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.707283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ff6ca-56a4-4b20-9a0c-ef564f0fbdf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.708176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ff6ca-56a4-4b20-9a0c-ef564f0fbdf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.711364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ff6ca-56a4-4b20-9a0c-ef564f0fbdf0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.716814 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1033/2084] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:09 2022] POST /identity/v3/users => generated 297 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:09.724782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae666005-e0b5-420f-9c84-da2d69d66a25 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.726428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1052/2085] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:09.730475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ff6ca-56a4-4b20-9a0c-ef564f0fbdf0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.740316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae666005-e0b5-420f-9c84-da2d69d66a25 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.743074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ff6ca-56a4-4b20-9a0c-ef564f0fbdf0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.746126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740078f2-08d0-4c84-8b00-0ba28444ea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.746934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740078f2-08d0-4c84-8b00-0ba28444ea39 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.747327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740078f2-08d0-4c84-8b00-0ba28444ea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.747668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740078f2-08d0-4c84-8b00-0ba28444ea39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.749297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740078f2-08d0-4c84-8b00-0ba28444ea39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.749461 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bff4c1-e0f0-4889-ae75-cfbb5a1f0890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.750139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bff4c1-e0f0-4889-ae75-cfbb5a1f0890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.750430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bff4c1-e0f0-4889-ae75-cfbb5a1f0890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.750708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bff4c1-e0f0-4889-ae75-cfbb5a1f0890 admin admin] PATH_INFO: `/v3/projects/d58778f8a4e74c71b4dee0724129fbb8/users/f759de0878954925a0c159f884c01590/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.752038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740078f2-08d0-4c84-8b00-0ba28444ea39 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.769751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1034/2086] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:09.771734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bff4c1-e0f0-4889-ae75-cfbb5a1f0890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58778f8a4e74c71b4dee0724129fbb8, user_id=f759de0878954925a0c159f884c01590, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.774679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bff4c1-e0f0-4889-ae75-cfbb5a1f0890 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.776028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740078f2-08d0-4c84-8b00-0ba28444ea39 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.791800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740078f2-08d0-4c84-8b00-0ba28444ea39 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.793228 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1053/2087] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:09 2022] PUT /identity/v3/projects/d58778f8a4e74c71b4dee0724129fbb8/users/f759de0878954925a0c159f884c01590/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:01:09.798019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bff4c1-e0f0-4889-ae75-cfbb5a1f0890 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.866381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633e9eb-fb28-4172-b972-15f26455ad27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.867099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633e9eb-fb28-4172-b972-15f26455ad27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.867388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633e9eb-fb28-4172-b972-15f26455ad27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.867670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633e9eb-fb28-4172-b972-15f26455ad27 admin admin] PATH_INFO: `/v3/projects/71ac130acb9f4543b97d752ca6b9ea4e/users/ffb7cbf0b8ab4a75a1708722fa31fee9/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.884916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bff4c1-e0f0-4889-ae75-cfbb5a1f0890 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.885572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633e9eb-fb28-4172-b972-15f26455ad27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ac130acb9f4543b97d752ca6b9ea4e, user_id=ffb7cbf0b8ab4a75a1708722fa31fee9, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.888333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633e9eb-fb28-4172-b972-15f26455ad27 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.890323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c6db7-6829-4803-8fcf-191117df2ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.891003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6db7-6829-4803-8fcf-191117df2ada admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.891289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6db7-6829-4803-8fcf-191117df2ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.891568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6db7-6829-4803-8fcf-191117df2ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.892539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c6db7-6829-4803-8fcf-191117df2ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.893876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c6db7-6829-4803-8fcf-191117df2ada admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.907984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1035/2088] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:09 2022] PUT /identity/v3/projects/71ac130acb9f4543b97d752ca6b9ea4e/users/ffb7cbf0b8ab4a75a1708722fa31fee9/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:09.908156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1054/2089] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:09.909050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633e9eb-fb28-4172-b972-15f26455ad27 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.909203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c6db7-6829-4803-8fcf-191117df2ada admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:09.921219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c6db7-6829-4803-8fcf-191117df2ada admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.963568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:09.979126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633e9eb-fb28-4172-b972-15f26455ad27 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:09.985375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b190c3-69ac-45ad-a8c4-ac8763015568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.986006 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b190c3-69ac-45ad-a8c4-ac8763015568 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.986573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b190c3-69ac-45ad-a8c4-ac8763015568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.986908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b190c3-69ac-45ad-a8c4-ac8763015568 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.988097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b190c3-69ac-45ad-a8c4-ac8763015568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.989784 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b190c3-69ac-45ad-a8c4-ac8763015568 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:09.993378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a691e4c-359a-4f4d-bcd8-1e68e1fcccd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:09.994184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a691e4c-359a-4f4d-bcd8-1e68e1fcccd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:09.994468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a691e4c-359a-4f4d-bcd8-1e68e1fcccd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:09.994745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a691e4c-359a-4f4d-bcd8-1e68e1fcccd9 admin admin] PATH_INFO: `/v3/projects/d58778f8a4e74c71b4dee0724129fbb8/users/f759de0878954925a0c159f884c01590/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:09.996756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a691e4c-359a-4f4d-bcd8-1e68e1fcccd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58778f8a4e74c71b4dee0724129fbb8, user_id=f759de0878954925a0c159f884c01590, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:09.999706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a691e4c-359a-4f4d-bcd8-1e68e1fcccd9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:10.000187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1036/2090] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:10.004341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b190c3-69ac-45ad-a8c4-ac8763015568 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:10.010618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1055/2091] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:09 2022] PUT /identity/v3/projects/d58778f8a4e74c71b4dee0724129fbb8/users/f759de0878954925a0c159f884c01590/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:10.014903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b190c3-69ac-45ad-a8c4-ac8763015568 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:10.022566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298977d-12e4-4109-a496-d61c012e253b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:10.022885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298977d-12e4-4109-a496-d61c012e253b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:10.023166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298977d-12e4-4109-a496-d61c012e253b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:10.052203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7298977d-12e4-4109-a496-d61c012e253b None None] MFA Rules not processed for user `f759de0878954925a0c159f884c01590`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:10.053448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7298977d-12e4-4109-a496-d61c012e253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:10.071510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf117acc-1cf2-45d3-99cb-c891f8abff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:10.072190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf117acc-1cf2-45d3-99cb-c891f8abff8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:10.072470 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf117acc-1cf2-45d3-99cb-c891f8abff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:10.072682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:10.072822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf117acc-1cf2-45d3-99cb-c891f8abff8e admin admin] PATH_INFO: `/v3/projects/71ac130acb9f4543b97d752ca6b9ea4e/users/ffb7cbf0b8ab4a75a1708722fa31fee9/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:10.074704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf117acc-1cf2-45d3-99cb-c891f8abff8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ac130acb9f4543b97d752ca6b9ea4e, user_id=ffb7cbf0b8ab4a75a1708722fa31fee9, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:10.078938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf117acc-1cf2-45d3-99cb-c891f8abff8e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:10.094961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1037/2092] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:10 2022] PUT /identity/v3/projects/71ac130acb9f4543b97d752ca6b9ea4e/users/ffb7cbf0b8ab4a75a1708722fa31fee9/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:01:10.104877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d698a0c-c424-4d45-b136-592063af2f02 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:10.105162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d698a0c-c424-4d45-b136-592063af2f02 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:10.105457 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d698a0c-c424-4d45-b136-592063af2f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:10.126255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1056/2093] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:10.133138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7d698a0c-c424-4d45-b136-592063af2f02 None None] MFA Rules not processed for user `ffb7cbf0b8ab4a75a1708722fa31fee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:10.134320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7d698a0c-c424-4d45-b136-592063af2f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:10.137514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298977d-12e4-4109-a496-d61c012e253b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:10.139692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7298977d-12e4-4109-a496-d61c012e253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:10.200857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298977d-12e4-4109-a496-d61c012e253b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:10.202969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1038/2094] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:10.213921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d698a0c-c424-4d45-b136-592063af2f02 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:10.215485 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7d698a0c-c424-4d45-b136-592063af2f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:10.252766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fd99a-e913-4bea-ba29-e8c2974e1685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:10.253390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fd99a-e913-4bea-ba29-e8c2974e1685 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:10.253673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fd99a-e913-4bea-ba29-e8c2974e1685 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:10.253950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fd99a-e913-4bea-ba29-e8c2974e1685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:10.277761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d698a0c-c424-4d45-b136-592063af2f02 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:10.282101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a164581-b54c-4ab5-b77f-134e863d79fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:10.282714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a164581-b54c-4ab5-b77f-134e863d79fe service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:10.283024 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a164581-b54c-4ab5-b77f-134e863d79fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:10.283298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a164581-b54c-4ab5-b77f-134e863d79fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:10.292900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a164581-b54c-4ab5-b77f-134e863d79fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:10.294323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a164581-b54c-4ab5-b77f-134e863d79fe service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:10.303979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1039/2095] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:10.317554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fd99a-e913-4bea-ba29-e8c2974e1685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:10.319034 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fd99a-e913-4bea-ba29-e8c2974e1685 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:10.328808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1057/2096] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:10 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:01:10.412448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a164581-b54c-4ab5-b77f-134e863d79fe service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:10.422832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a164581-b54c-4ab5-b77f-134e863d79fe service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:10.426755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea70e0-2c1a-4215-96af-4e5baa71aa2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:10.427403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea70e0-2c1a-4215-96af-4e5baa71aa2c service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:10.427686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea70e0-2c1a-4215-96af-4e5baa71aa2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:10.427963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea70e0-2c1a-4215-96af-4e5baa71aa2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:10.435439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:10.493577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea70e0-2c1a-4215-96af-4e5baa71aa2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:10.495127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea70e0-2c1a-4215-96af-4e5baa71aa2c service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:10.505808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1040/2097] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:10.757967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fd99a-e913-4bea-ba29-e8c2974e1685 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:10.792315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:10.845093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fd99a-e913-4bea-ba29-e8c2974e1685 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:10.846602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:10.847468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:10.847776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:10.848117 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] PATH_INFO: `/v3/users/0f59f99c000c4cbeb36628a8e6c21f31` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:10.849384 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f59f99c000c4cbeb36628a8e6c21f31)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:10.851545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:10.911860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f59f99c000c4cbeb36628a8e6c21f31'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:10.913206 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f59f99c000c4cbeb36628a8e6c21f31'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:10.929744 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f59f99c000c4cbeb36628a8e6c21f31'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:10.937649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f59f99c000c4cbeb36628a8e6c21f31'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:10.945654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f59f99c000c4cbeb36628a8e6c21f31'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:10.955498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1058/2098] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:10 2022] DELETE /identity/v3/users/0f59f99c000c4cbeb36628a8e6c21f31 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:10.959898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc666eb-9958-450e-920a-db812254bcc2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:10.960340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc666eb-9958-450e-920a-db812254bcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:10.960451 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc666eb-9958-450e-920a-db812254bcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:10.979703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7dc666eb-9958-450e-920a-db812254bcc2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:10.980968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7dc666eb-9958-450e-920a-db812254bcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:11.054884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1041/2099] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:11.062516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:11.064003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:11.124806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978285e-ddb5-4230-9939-cea1e341f6b6 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:11.128468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d221695-c747-4bbb-82cc-a58ca6d42aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:11.129086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221695-c747-4bbb-82cc-a58ca6d42aba service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:11.129371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221695-c747-4bbb-82cc-a58ca6d42aba service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:11.129646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221695-c747-4bbb-82cc-a58ca6d42aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:11.139727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d221695-c747-4bbb-82cc-a58ca6d42aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:11.141181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d221695-c747-4bbb-82cc-a58ca6d42aba service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:11.151565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1059/2100] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:11.416524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc666eb-9958-450e-920a-db812254bcc2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:11.418001 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7dc666eb-9958-450e-920a-db812254bcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:11.431571 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc666eb-9958-450e-920a-db812254bcc2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:11.437472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:11.438221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:11.438613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:11.438983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] PATH_INFO: `/v3/projects/c23618da05ef412380473903e08a2109` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:11.440221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c23618da05ef412380473903e08a2109)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:11.442204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:11.489742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c23618da05ef412380473903e08a2109', 'request_id': 'req-ab15af20-2fc5-454d-8639-8b34b376b733'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:11.495528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c23618da05ef412380473903e08a2109', 'request_id': 'req-ab15af20-2fc5-454d-8639-8b34b376b733'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:11.503324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c23618da05ef412380473903e08a2109', 'request_id': 'req-ab15af20-2fc5-454d-8639-8b34b376b733'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:11.515012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] The token cache is being invalidate because project c23618da05ef412380473903e08a2109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:11.515327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:11.516069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:11.517572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1042/2101] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:11 2022] DELETE /identity/v3/projects/c23618da05ef412380473903e08a2109 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:11.521524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d221695-c747-4bbb-82cc-a58ca6d42aba service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:11.531273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d221695-c747-4bbb-82cc-a58ca6d42aba service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:11.575306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:11.592282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163f075-771c-454e-87db-0cc8be0f1550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:11.592920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163f075-771c-454e-87db-0cc8be0f1550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:11.593205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163f075-771c-454e-87db-0cc8be0f1550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:11.593480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163f075-771c-454e-87db-0cc8be0f1550 admin admin] PATH_INFO: `/v3/users/f759de0878954925a0c159f884c01590` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:11.594686 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163f075-771c-454e-87db-0cc8be0f1550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f759de0878954925a0c159f884c01590)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:11.596794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163f075-771c-454e-87db-0cc8be0f1550 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:11.646792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1163f075-771c-454e-87db-0cc8be0f1550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f759de0878954925a0c159f884c01590'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:11.647636 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1163f075-771c-454e-87db-0cc8be0f1550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f759de0878954925a0c159f884c01590'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:11.661340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1163f075-771c-454e-87db-0cc8be0f1550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f759de0878954925a0c159f884c01590'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:11.669309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1163f075-771c-454e-87db-0cc8be0f1550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f759de0878954925a0c159f884c01590'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:11.676612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1163f075-771c-454e-87db-0cc8be0f1550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f759de0878954925a0c159f884c01590'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:11.683155 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1060/2102] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:11 2022] DELETE /identity/v3/users/f759de0878954925a0c159f884c01590 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:12.004930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:12.006485 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:12.022454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15af20-2fc5-454d-8639-8b34b376b733 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:12.078427 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:12.125873 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:12.126526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:12.126841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:12.127125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] PATH_INFO: `/v3/projects/d58778f8a4e74c71b4dee0724129fbb8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:12.128319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d58778f8a4e74c71b4dee0724129fbb8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:12.130229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:12.188295 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd58778f8a4e74c71b4dee0724129fbb8', 'request_id': 'req-04e55d0b-65f6-488d-8ed0-75549238f298'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:12.194674 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd58778f8a4e74c71b4dee0724129fbb8', 'request_id': 'req-04e55d0b-65f6-488d-8ed0-75549238f298'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:12.205255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd58778f8a4e74c71b4dee0724129fbb8', 'request_id': 'req-04e55d0b-65f6-488d-8ed0-75549238f298'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:12.219583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] The token cache is being invalidate because project d58778f8a4e74c71b4dee0724129fbb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:12.219965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:12.220720 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-04e55d0b-65f6-488d-8ed0-75549238f298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:12.222678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1043/2103] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:12 2022] DELETE /identity/v3/projects/d58778f8a4e74c71b4dee0724129fbb8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:12.227250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243aeaf7-c3f8-4710-89e8-e94f67a4f2c7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:12.227736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243aeaf7-c3f8-4710-89e8-e94f67a4f2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:12.228041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243aeaf7-c3f8-4710-89e8-e94f67a4f2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:12.252040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-243aeaf7-c3f8-4710-89e8-e94f67a4f2c7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:12.253480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-243aeaf7-c3f8-4710-89e8-e94f67a4f2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:12.338963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1061/2104] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:12.343782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087ada8-25d6-415d-b49c-28a85312d255 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:12.344072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087ada8-25d6-415d-b49c-28a85312d255 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:12.344407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087ada8-25d6-415d-b49c-28a85312d255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:12.373350 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0087ada8-25d6-415d-b49c-28a85312d255 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:12.374725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0087ada8-25d6-415d-b49c-28a85312d255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:12.383402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1044/2105] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:12.387970 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243aeaf7-c3f8-4710-89e8-e94f67a4f2c7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:12.400065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243aeaf7-c3f8-4710-89e8-e94f67a4f2c7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:12.406748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e094920-f439-4234-bd72-3df23e3c470e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:12.407407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e094920-f439-4234-bd72-3df23e3c470e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:12.407690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e094920-f439-4234-bd72-3df23e3c470e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:12.407964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e094920-f439-4234-bd72-3df23e3c470e admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:12.409702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e094920-f439-4234-bd72-3df23e3c470e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:12.410047 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e094920-f439-4234-bd72-3df23e3c470e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:12.411690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e094920-f439-4234-bd72-3df23e3c470e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:12.424907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1062/2106] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:01:12.429462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087ada8-25d6-415d-b49c-28a85312d255 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:12.440675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087ada8-25d6-415d-b49c-28a85312d255 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:12.445538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e62b76-2c1f-40c7-b89d-2a364a0ee391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:12.446259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62b76-2c1f-40c7-b89d-2a364a0ee391 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:12.446542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62b76-2c1f-40c7-b89d-2a364a0ee391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:12.446899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62b76-2c1f-40c7-b89d-2a364a0ee391 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:12.447563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e62b76-2c1f-40c7-b89d-2a364a0ee391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:12.449479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e62b76-2c1f-40c7-b89d-2a364a0ee391 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:12.483560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1045/2107] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:12 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:12.487770 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e094920-f439-4234-bd72-3df23e3c470e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:12.501662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e094920-f439-4234-bd72-3df23e3c470e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:12.564008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:12.579486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff3bc0-d1a9-4238-87bd-86322cd2d264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:12.581092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff3bc0-d1a9-4238-87bd-86322cd2d264 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:12.581523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff3bc0-d1a9-4238-87bd-86322cd2d264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:12.581944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff3bc0-d1a9-4238-87bd-86322cd2d264 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:12.583084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff3bc0-d1a9-4238-87bd-86322cd2d264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:12.585154 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff3bc0-d1a9-4238-87bd-86322cd2d264 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:12.610210 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1063/2108] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:12 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:12.614237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e62b76-2c1f-40c7-b89d-2a364a0ee391 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:12.626119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e62b76-2c1f-40c7-b89d-2a364a0ee391 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:12.631395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76ddbf-9995-4046-b3ad-028eedff3712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:12.631959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76ddbf-9995-4046-b3ad-028eedff3712 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:12.632230 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76ddbf-9995-4046-b3ad-028eedff3712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:12.632510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76ddbf-9995-4046-b3ad-028eedff3712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:12.633218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76ddbf-9995-4046-b3ad-028eedff3712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:12.634989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76ddbf-9995-4046-b3ad-028eedff3712 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:12.650272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1046/2109] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:12.651529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff3bc0-d1a9-4238-87bd-86322cd2d264 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:12.663277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff3bc0-d1a9-4238-87bd-86322cd2d264 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:12.668935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de9666-06f9-412d-bcf3-1015db9e7cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:12.669669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de9666-06f9-412d-bcf3-1015db9e7cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:12.670023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de9666-06f9-412d-bcf3-1015db9e7cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:12.670387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de9666-06f9-412d-bcf3-1015db9e7cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:12.671351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de9666-06f9-412d-bcf3-1015db9e7cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:12.673038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de9666-06f9-412d-bcf3-1015db9e7cb8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:12.685710 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1064/2110] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:12.691197 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76ddbf-9995-4046-b3ad-028eedff3712 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:12.709369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76ddbf-9995-4046-b3ad-028eedff3712 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:12.716000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47810363-3b42-41f5-97a2-c7cc019f6027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:12.717072 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47810363-3b42-41f5-97a2-c7cc019f6027 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:12.717797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47810363-3b42-41f5-97a2-c7cc019f6027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:12.718423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47810363-3b42-41f5-97a2-c7cc019f6027 admin admin] PATH_INFO: `/v3/projects/3c4e77e362704c7dbbe5f55b08251629/users/051869af37d64ba487aa3f97f9824cfd/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:12.739256 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47810363-3b42-41f5-97a2-c7cc019f6027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4e77e362704c7dbbe5f55b08251629, user_id=051869af37d64ba487aa3f97f9824cfd, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:12.742483 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47810363-3b42-41f5-97a2-c7cc019f6027 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:12.769083 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1047/2111] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:12 2022] PUT /identity/v3/projects/3c4e77e362704c7dbbe5f55b08251629/users/051869af37d64ba487aa3f97f9824cfd/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:12.781776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69aec5-0bec-4928-976d-f267a1fbeef1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:12.782141 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69aec5-0bec-4928-976d-f267a1fbeef1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:12.782428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69aec5-0bec-4928-976d-f267a1fbeef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:12.818673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4a69aec5-0bec-4928-976d-f267a1fbeef1 None None] MFA Rules not processed for user `051869af37d64ba487aa3f97f9824cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:12.820448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4a69aec5-0bec-4928-976d-f267a1fbeef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:12.902828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1065/2112] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:12.904294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47810363-3b42-41f5-97a2-c7cc019f6027 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:12.978633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:12.985446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47810363-3b42-41f5-97a2-c7cc019f6027 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:12.994376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b51b3f-75bd-4229-8bfa-e6f577ef5cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:12.995457 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b51b3f-75bd-4229-8bfa-e6f577ef5cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:12.995852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b51b3f-75bd-4229-8bfa-e6f577ef5cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:12.996203 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b51b3f-75bd-4229-8bfa-e6f577ef5cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:12.997350 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b51b3f-75bd-4229-8bfa-e6f577ef5cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:13.001344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b51b3f-75bd-4229-8bfa-e6f577ef5cf1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:13.040659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1048/2113] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:12 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:13.042219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69aec5-0bec-4928-976d-f267a1fbeef1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:13.057234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69aec5-0bec-4928-976d-f267a1fbeef1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:13.131454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff5a70-fc3d-409f-abd7-d4785e1f147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:13.132259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff5a70-fc3d-409f-abd7-d4785e1f147b admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:13.132788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff5a70-fc3d-409f-abd7-d4785e1f147b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:13.133206 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff5a70-fc3d-409f-abd7-d4785e1f147b admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:13.134160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff5a70-fc3d-409f-abd7-d4785e1f147b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:13.136382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff5a70-fc3d-409f-abd7-d4785e1f147b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:13.168434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1066/2114] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:13 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:13.172374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b51b3f-75bd-4229-8bfa-e6f577ef5cf1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:13.185863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b51b3f-75bd-4229-8bfa-e6f577ef5cf1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:13.191811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984a565-689f-4092-8f80-09ddc0d736bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:13.192583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984a565-689f-4092-8f80-09ddc0d736bc admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:13.192924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984a565-689f-4092-8f80-09ddc0d736bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:13.193251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984a565-689f-4092-8f80-09ddc0d736bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:13.193963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984a565-689f-4092-8f80-09ddc0d736bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:13.195766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984a565-689f-4092-8f80-09ddc0d736bc admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:13.207380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff5a70-fc3d-409f-abd7-d4785e1f147b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:13.209440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b8ff5a70-fc3d-409f-abd7-d4785e1f147b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:13.233959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff5a70-fc3d-409f-abd7-d4785e1f147b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:13.236694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1049/2115] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:13.236694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984a565-689f-4092-8f80-09ddc0d736bc admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:13.249251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5e017-fc20-4221-9e21-6fac80bf97a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:13.249925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5e017-fc20-4221-9e21-6fac80bf97a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:13.250214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5e017-fc20-4221-9e21-6fac80bf97a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:13.250492 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5e017-fc20-4221-9e21-6fac80bf97a8 admin admin] PATH_INFO: `/v3/users/c604ee24070f4349805ba9311926f87a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:13.252754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5e017-fc20-4221-9e21-6fac80bf97a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c604ee24070f4349805ba9311926f87a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:13.253928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5e017-fc20-4221-9e21-6fac80bf97a8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:13.256740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984a565-689f-4092-8f80-09ddc0d736bc admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:13.266455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13682305-6e5c-4a4e-a30d-b90cbc54789d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:13.267233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13682305-6e5c-4a4e-a30d-b90cbc54789d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:13.267502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13682305-6e5c-4a4e-a30d-b90cbc54789d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:13.267771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13682305-6e5c-4a4e-a30d-b90cbc54789d admin admin] PATH_INFO: `/v3/projects/bd659182a7b7425e9dae36d3192f9b5d/users/bf83cb991956450e94317da9b6211c6c/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:13.294968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13682305-6e5c-4a4e-a30d-b90cbc54789d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd659182a7b7425e9dae36d3192f9b5d, user_id=bf83cb991956450e94317da9b6211c6c, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:13.298156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13682305-6e5c-4a4e-a30d-b90cbc54789d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:13.315352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1050/2116] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:13 2022] PUT /identity/v3/projects/bd659182a7b7425e9dae36d3192f9b5d/users/bf83cb991956450e94317da9b6211c6c/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:13.319710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13682305-6e5c-4a4e-a30d-b90cbc54789d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:13.333618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:13.341609 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a9e5e017-fc20-4221-9e21-6fac80bf97a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c604ee24070f4349805ba9311926f87a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:13.343151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a9e5e017-fc20-4221-9e21-6fac80bf97a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c604ee24070f4349805ba9311926f87a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:13.363618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a9e5e017-fc20-4221-9e21-6fac80bf97a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c604ee24070f4349805ba9311926f87a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:13.377015 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a9e5e017-fc20-4221-9e21-6fac80bf97a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c604ee24070f4349805ba9311926f87a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:13.401628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a9e5e017-fc20-4221-9e21-6fac80bf97a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c604ee24070f4349805ba9311926f87a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:13.413797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1067/2117] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:13 2022] DELETE /identity/v3/users/c604ee24070f4349805ba9311926f87a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:13.418161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940744d6-538a-42d3-94da-c86677f2bee8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:13.418455 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940744d6-538a-42d3-94da-c86677f2bee8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:13.418734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940744d6-538a-42d3-94da-c86677f2bee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:13.421168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13682305-6e5c-4a4e-a30d-b90cbc54789d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:13.472675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-940744d6-538a-42d3-94da-c86677f2bee8 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:13.474118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-940744d6-538a-42d3-94da-c86677f2bee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:13.530313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc9423-bd1a-4824-9d17-c330471e5852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:13.531879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc9423-bd1a-4824-9d17-c330471e5852 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:13.532543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc9423-bd1a-4824-9d17-c330471e5852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:13.533074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc9423-bd1a-4824-9d17-c330471e5852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:13.534692 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc9423-bd1a-4824-9d17-c330471e5852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:13.537071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc9423-bd1a-4824-9d17-c330471e5852 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:13.551151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1051/2118] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:13.555346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc9423-bd1a-4824-9d17-c330471e5852 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:13.559550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-efdc9423-bd1a-4824-9d17-c330471e5852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:13.572856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1068/2119] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:13.575466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc9423-bd1a-4824-9d17-c330471e5852 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:13.580642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940744d6-538a-42d3-94da-c86677f2bee8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:13.581570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305247d4-e8d5-4c86-9b12-475d0f9897f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:13.582057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-940744d6-538a-42d3-94da-c86677f2bee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:13.582401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305247d4-e8d5-4c86-9b12-475d0f9897f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:13.582708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305247d4-e8d5-4c86-9b12-475d0f9897f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:13.583020 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305247d4-e8d5-4c86-9b12-475d0f9897f9 admin admin] PATH_INFO: `/v3/projects/bd659182a7b7425e9dae36d3192f9b5d/users/bf83cb991956450e94317da9b6211c6c/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:13.585093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305247d4-e8d5-4c86-9b12-475d0f9897f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd659182a7b7425e9dae36d3192f9b5d, user_id=bf83cb991956450e94317da9b6211c6c, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:13.588907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305247d4-e8d5-4c86-9b12-475d0f9897f9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:13.605443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1052/2120] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:13 2022] PUT /identity/v3/projects/bd659182a7b7425e9dae36d3192f9b5d/users/bf83cb991956450e94317da9b6211c6c/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:13.613849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b0fe-c103-4f89-aaec-5a6bcf4c2725 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:13.614234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b0fe-c103-4f89-aaec-5a6bcf4c2725 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:13.614548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b0fe-c103-4f89-aaec-5a6bcf4c2725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:13.655532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c916b0fe-c103-4f89-aaec-5a6bcf4c2725 None None] MFA Rules not processed for user `bf83cb991956450e94317da9b6211c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:13.656881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c916b0fe-c103-4f89-aaec-5a6bcf4c2725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:13.665424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940744d6-538a-42d3-94da-c86677f2bee8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:13.691046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:13.730325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64758e-1d37-461d-90b3-59f66f01859f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:13.730995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64758e-1d37-461d-90b3-59f66f01859f service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:13.731277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64758e-1d37-461d-90b3-59f66f01859f service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:13.731552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64758e-1d37-461d-90b3-59f66f01859f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:13.749716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1053/2121] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:13.761988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916b0fe-c103-4f89-aaec-5a6bcf4c2725 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:13.763396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c916b0fe-c103-4f89-aaec-5a6bcf4c2725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:13.769830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:13.808398 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64758e-1d37-461d-90b3-59f66f01859f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:13.809912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64758e-1d37-461d-90b3-59f66f01859f service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:13.821144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1069/2122] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:01:13.840815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916b0fe-c103-4f89-aaec-5a6bcf4c2725 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:13.845059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e8cb6-e705-4aed-8150-9f1fcd37d799 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:13.845684 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e8cb6-e705-4aed-8150-9f1fcd37d799 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:13.845967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e8cb6-e705-4aed-8150-9f1fcd37d799 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:13.846243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e8cb6-e705-4aed-8150-9f1fcd37d799 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:13.855949 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e8cb6-e705-4aed-8150-9f1fcd37d799 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:13.857403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e8cb6-e705-4aed-8150-9f1fcd37d799 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:13.868963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1054/2123] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:01:14.130625 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64758e-1d37-461d-90b3-59f66f01859f service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:14.132620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4e64758e-1d37-461d-90b3-59f66f01859f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:14.156086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64758e-1d37-461d-90b3-59f66f01859f service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:14.162471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:14.163538 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:14.163822 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:14.164099 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] PATH_INFO: `/v3/projects/043054133ca246f7a0ca0fbdf3fd6707` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:14.165264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] RBAC: Authorizing `identity:delete_project(project_id=043054133ca246f7a0ca0fbdf3fd6707)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:14.167221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:14.221423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '043054133ca246f7a0ca0fbdf3fd6707', 'request_id': 'req-8622c231-52b7-4ab7-af79-f784f0b7c40e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.227654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '043054133ca246f7a0ca0fbdf3fd6707', 'request_id': 'req-8622c231-52b7-4ab7-af79-f784f0b7c40e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.233602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e8cb6-e705-4aed-8150-9f1fcd37d799 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:14.234083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '043054133ca246f7a0ca0fbdf3fd6707', 'request_id': 'req-8622c231-52b7-4ab7-af79-f784f0b7c40e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.245865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e8cb6-e705-4aed-8150-9f1fcd37d799 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:14.246177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] The token cache is being invalidate because project 043054133ca246f7a0ca0fbdf3fd6707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:14.246469 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.247148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.249622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1070/2124] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:14 2022] DELETE /identity/v3/projects/043054133ca246f7a0ca0fbdf3fd6707 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:14.253594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:14.255031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:14.306377 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97fe7b-1dcf-4e0c-94b1-7fc9d8134e70 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf83cb991956450e94317da9b6211c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd659182a7b7425e9dae36d3192f9b5d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:14.307064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97fe7b-1dcf-4e0c-94b1-7fc9d8134e70 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:14.307358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97fe7b-1dcf-4e0c-94b1-7fc9d8134e70 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:14.307644 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97fe7b-1dcf-4e0c-94b1-7fc9d8134e70 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:14.308393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97fe7b-1dcf-4e0c-94b1-7fc9d8134e70 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:14.308670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97fe7b-1dcf-4e0c-94b1-7fc9d8134e70 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:14.310418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97fe7b-1dcf-4e0c-94b1-7fc9d8134e70 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:14.323370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622c231-52b7-4ab7-af79-f784f0b7c40e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:14.323578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1055/2125] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:01:14.327975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97fe7b-1dcf-4e0c-94b1-7fc9d8134e70 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:14.329666 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cb97fe7b-1dcf-4e0c-94b1-7fc9d8134e70 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:14.329952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:14.330593 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:14.330914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:14.331250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] PATH_INFO: `/v3/users/ffb7cbf0b8ab4a75a1708722fa31fee9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:14.332408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb7cbf0b8ab4a75a1708722fa31fee9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:14.334381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:14.342816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97fe7b-1dcf-4e0c-94b1-7fc9d8134e70 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:14.347725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d7e6b-1908-4bb1-9e35-f35ef75ead5f tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf83cb991956450e94317da9b6211c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd659182a7b7425e9dae36d3192f9b5d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:14.348343 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d7e6b-1908-4bb1-9e35-f35ef75ead5f tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:14.348629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d7e6b-1908-4bb1-9e35-f35ef75ead5f tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:14.348908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d7e6b-1908-4bb1-9e35-f35ef75ead5f tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:14.349577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d7e6b-1908-4bb1-9e35-f35ef75ead5f tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:14.351525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d7e6b-1908-4bb1-9e35-f35ef75ead5f tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:14.384679 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1056/2126] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:14 2022] POST /identity/v3/projects => generated 355 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:14.385427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb7cbf0b8ab4a75a1708722fa31fee9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.386081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb7cbf0b8ab4a75a1708722fa31fee9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.388375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d7e6b-1908-4bb1-9e35-f35ef75ead5f tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:14.391391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:14.399986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb7cbf0b8ab4a75a1708722fa31fee9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.408899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb7cbf0b8ab4a75a1708722fa31fee9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.418189 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb7cbf0b8ab4a75a1708722fa31fee9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.425025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1071/2127] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:14 2022] DELETE /identity/v3/users/ffb7cbf0b8ab4a75a1708722fa31fee9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:01:14.432850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:14.459976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d7e6b-1908-4bb1-9e35-f35ef75ead5f tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:14.524748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b62ba-5d23-4b61-823a-9fa02d783a73 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf83cb991956450e94317da9b6211c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd659182a7b7425e9dae36d3192f9b5d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:14.525482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b62ba-5d23-4b61-823a-9fa02d783a73 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:14.525964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b62ba-5d23-4b61-823a-9fa02d783a73 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:14.526328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b62ba-5d23-4b61-823a-9fa02d783a73 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:14.527766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b62ba-5d23-4b61-823a-9fa02d783a73 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:14.528193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b62ba-5d23-4b61-823a-9fa02d783a73 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:14.530837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b62ba-5d23-4b61-823a-9fa02d783a73 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:14.545760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1057/2128] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:01:14.594855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:14.596757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:14.626371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b8ac1-aed4-41bb-8010-2cdfcc54ba58 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:14.631392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf83cb991956450e94317da9b6211c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd659182a7b7425e9dae36d3192f9b5d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:14.632047 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:14.632334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:14.632749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] PATH_INFO: `/v3/projects/93226a819807499a9c4570659bd41a3f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:14.633934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: Authorizing `identity:delete_project(project_id=93226a819807499a9c4570659bd41a3f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:14.636473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:14.705276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93226a819807499a9c4570659bd41a3f', 'request_id': 'req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.712304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93226a819807499a9c4570659bd41a3f', 'request_id': 'req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.723291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93226a819807499a9c4570659bd41a3f', 'request_id': 'req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.740956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b62ba-5d23-4b61-823a-9fa02d783a73 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:14.743225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-419b62ba-5d23-4b61-823a-9fa02d783a73 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:14.744754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] The token cache is being invalidate because project 93226a819807499a9c4570659bd41a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:14.745054 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.746026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.747764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1072/2129] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:14 2022] DELETE /identity/v3/projects/93226a819807499a9c4570659bd41a3f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:14.827213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b62ba-5d23-4b61-823a-9fa02d783a73 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:14.908907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:14.909592 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:14.909849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:14.910171 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] PATH_INFO: `/v3/projects/71ac130acb9f4543b97d752ca6b9ea4e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:14.911428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71ac130acb9f4543b97d752ca6b9ea4e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:14.913677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:14.947504 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:14.970540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:14.971811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71ac130acb9f4543b97d752ca6b9ea4e', 'request_id': 'req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.972005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:14.977387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71ac130acb9f4543b97d752ca6b9ea4e', 'request_id': 'req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.985146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71ac130acb9f4543b97d752ca6b9ea4e', 'request_id': 'req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:14.985774 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db60ab5-30e0-44a4-93c4-fa0476a6fee6 tempest-VolumeQuotaClassesTest-1934348677 tempest-VolumeQuotaClassesTest-1934348677-project-admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:15.000500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] The token cache is being invalidate because project 71ac130acb9f4543b97d752ca6b9ea4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:15.000773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:15.001454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-221f20aa-dbdd-43b4-939c-7c2171e7c0b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:15.003007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1058/2130] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:14 2022] DELETE /identity/v3/projects/71ac130acb9f4543b97d752ca6b9ea4e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:01:15.007670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd85edb-bd06-4aac-ba9f-c74fa0f0af73 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.008110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd85edb-bd06-4aac-ba9f-c74fa0f0af73 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.008390 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd85edb-bd06-4aac-ba9f-c74fa0f0af73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.027719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bbd85edb-bd06-4aac-ba9f-c74fa0f0af73 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:15.028946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bbd85edb-bd06-4aac-ba9f-c74fa0f0af73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:15.052360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507df965-7398-4557-9525-7705872c5e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:15.053013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507df965-7398-4557-9525-7705872c5e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.053301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507df965-7398-4557-9525-7705872c5e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.053700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507df965-7398-4557-9525-7705872c5e31 admin admin] PATH_INFO: `/v3/users/051869af37d64ba487aa3f97f9824cfd` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.055129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507df965-7398-4557-9525-7705872c5e31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=051869af37d64ba487aa3f97f9824cfd)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:15.057046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507df965-7398-4557-9525-7705872c5e31 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:15.100698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1059/2131] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:15.110972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca90ca8-c170-4475-b8c7-f7dd5e9a9e63 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.111332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca90ca8-c170-4475-b8c7-f7dd5e9a9e63 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.111845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca90ca8-c170-4475-b8c7-f7dd5e9a9e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.118535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-507df965-7398-4557-9525-7705872c5e31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051869af37d64ba487aa3f97f9824cfd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:15.119865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-507df965-7398-4557-9525-7705872c5e31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051869af37d64ba487aa3f97f9824cfd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:15.128559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:15.134527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8ca90ca8-c170-4475-b8c7-f7dd5e9a9e63 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:15.136118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8ca90ca8-c170-4475-b8c7-f7dd5e9a9e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:15.141399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-507df965-7398-4557-9525-7705872c5e31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051869af37d64ba487aa3f97f9824cfd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:15.147574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1060/2132] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:15.151657 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca90ca8-c170-4475-b8c7-f7dd5e9a9e63 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:15.154741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-507df965-7398-4557-9525-7705872c5e31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051869af37d64ba487aa3f97f9824cfd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:15.169026 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca90ca8-c170-4475-b8c7-f7dd5e9a9e63 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:15.173815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-507df965-7398-4557-9525-7705872c5e31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051869af37d64ba487aa3f97f9824cfd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:15.179315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e035374-0f3f-405b-82e4-69dd80085f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:15.179948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e035374-0f3f-405b-82e4-69dd80085f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.180262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e035374-0f3f-405b-82e4-69dd80085f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.180506 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e035374-0f3f-405b-82e4-69dd80085f39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.181448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e035374-0f3f-405b-82e4-69dd80085f39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:15.181758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e035374-0f3f-405b-82e4-69dd80085f39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:15.185467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e035374-0f3f-405b-82e4-69dd80085f39 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:15.185873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1073/2133] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:14 2022] DELETE /identity/v3/users/051869af37d64ba487aa3f97f9824cfd => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:01:15.187550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823701c6-f55d-4ab0-bc43-839733adb0c4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.188003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823701c6-f55d-4ab0-bc43-839733adb0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.188433 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823701c6-f55d-4ab0-bc43-839733adb0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.202622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1061/2134] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:01:15.203794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e035374-0f3f-405b-82e4-69dd80085f39 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:15.206622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4e035374-0f3f-405b-82e4-69dd80085f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:15.216122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-823701c6-f55d-4ab0-bc43-839733adb0c4 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:15.218265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-823701c6-f55d-4ab0-bc43-839733adb0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:15.227416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e035374-0f3f-405b-82e4-69dd80085f39 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:15.289055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1074/2135] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:15.305558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823701c6-f55d-4ab0-bc43-839733adb0c4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:15.306955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-823701c6-f55d-4ab0-bc43-839733adb0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:15.307500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab77ce-68ed-4309-adce-80debdcb27e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:15.308247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab77ce-68ed-4309-adce-80debdcb27e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.308649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab77ce-68ed-4309-adce-80debdcb27e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.309023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab77ce-68ed-4309-adce-80debdcb27e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.309824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab77ce-68ed-4309-adce-80debdcb27e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:15.313316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab77ce-68ed-4309-adce-80debdcb27e5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:15.368362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1062/2136] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:15 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:15.369478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab77ce-68ed-4309-adce-80debdcb27e5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:15.386540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823701c6-f55d-4ab0-bc43-839733adb0c4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:15.395889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:15.445388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab77ce-68ed-4309-adce-80debdcb27e5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:15.450094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a7884-facf-4fa2-8f7c-6905391897fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:15.450713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a7884-facf-4fa2-8f7c-6905391897fe admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.451022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a7884-facf-4fa2-8f7c-6905391897fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.451301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a7884-facf-4fa2-8f7c-6905391897fe admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.452526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a7884-facf-4fa2-8f7c-6905391897fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:15.452729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9920d0-a00d-4491-b4a4-4e266ab78183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:15.453360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9920d0-a00d-4491-b4a4-4e266ab78183 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.453651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9920d0-a00d-4491-b4a4-4e266ab78183 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.453934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9920d0-a00d-4491-b4a4-4e266ab78183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.454536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a7884-facf-4fa2-8f7c-6905391897fe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:15.463331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9920d0-a00d-4491-b4a4-4e266ab78183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:15.464960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9920d0-a00d-4491-b4a4-4e266ab78183 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:15.476753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1075/2137] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:15.482326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1063/2138] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:15 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:15.487213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9920d0-a00d-4491-b4a4-4e266ab78183 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:15.496915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9920d0-a00d-4491-b4a4-4e266ab78183 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:15.502398 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f730b24-e35f-409d-835a-6d2d0b50014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:15.503055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f730b24-e35f-409d-835a-6d2d0b50014b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.503336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f730b24-e35f-409d-835a-6d2d0b50014b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.503704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f730b24-e35f-409d-835a-6d2d0b50014b admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.504616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f730b24-e35f-409d-835a-6d2d0b50014b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:15.506541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f730b24-e35f-409d-835a-6d2d0b50014b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:15.518827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1076/2139] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:15.522509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a7884-facf-4fa2-8f7c-6905391897fe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:15.537429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a7884-facf-4fa2-8f7c-6905391897fe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:15.539233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbd6f5-02e1-4552-a36b-2e98e3829f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:15.540002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbd6f5-02e1-4552-a36b-2e98e3829f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.540308 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbd6f5-02e1-4552-a36b-2e98e3829f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.541154 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbd6f5-02e1-4552-a36b-2e98e3829f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.541382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cbd6f5-02e1-4552-a36b-2e98e3829f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:15.546901 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cbd6f5-02e1-4552-a36b-2e98e3829f91 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:15.563814 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1064/2140] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:15.565827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f730b24-e35f-409d-835a-6d2d0b50014b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:15.577758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f730b24-e35f-409d-835a-6d2d0b50014b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:15.583300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e6f67-ec89-48de-8147-c4eb6cfa19c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:15.583983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e6f67-ec89-48de-8147-c4eb6cfa19c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.584263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e6f67-ec89-48de-8147-c4eb6cfa19c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.584539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e6f67-ec89-48de-8147-c4eb6cfa19c8 admin admin] PATH_INFO: `/v3/projects/c2f2154fa14b433f9beefe6148f62038/users/0d9203ca1da6465986c30444e92e6eed/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.602951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e6f67-ec89-48de-8147-c4eb6cfa19c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f2154fa14b433f9beefe6148f62038, user_id=0d9203ca1da6465986c30444e92e6eed, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:15.607501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e6f67-ec89-48de-8147-c4eb6cfa19c8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:15.610965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:15.624367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1077/2141] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:15 2022] PUT /identity/v3/projects/c2f2154fa14b433f9beefe6148f62038/users/0d9203ca1da6465986c30444e92e6eed/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:15.634459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ada56-ffaa-4362-a97a-c6321a9e20cf None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.634832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ada56-ffaa-4362-a97a-c6321a9e20cf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.635147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ada56-ffaa-4362-a97a-c6321a9e20cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.662044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734b51-8514-4c77-9f8e-bbcab5fa3002 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.662316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734b51-8514-4c77-9f8e-bbcab5fa3002 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.662595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734b51-8514-4c77-9f8e-bbcab5fa3002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.676287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-809ada56-ffaa-4362-a97a-c6321a9e20cf None None] MFA Rules not processed for user `0d9203ca1da6465986c30444e92e6eed`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:15.677420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-809ada56-ffaa-4362-a97a-c6321a9e20cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:15.690416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f9734b51-8514-4c77-9f8e-bbcab5fa3002 None None] MFA Rules not processed for user `820e385c27af49f08301be61fcde369b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:15.691896 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f9734b51-8514-4c77-9f8e-bbcab5fa3002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:15.756685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1078/2142] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:15.776862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1065/2143] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:15.786830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734b51-8514-4c77-9f8e-bbcab5fa3002 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:15.787038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f9734b51-8514-4c77-9f8e-bbcab5fa3002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:15.822116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ada56-ffaa-4362-a97a-c6321a9e20cf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:15.826317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-809ada56-ffaa-4362-a97a-c6321a9e20cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:15.865159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734b51-8514-4c77-9f8e-bbcab5fa3002 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:15.870421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714c03e-4fcf-436f-b6a4-350e4077df1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:15.871171 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714c03e-4fcf-436f-b6a4-350e4077df1c service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.871521 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714c03e-4fcf-436f-b6a4-350e4077df1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.871863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714c03e-4fcf-436f-b6a4-350e4077df1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.883254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714c03e-4fcf-436f-b6a4-350e4077df1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:15.885150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714c03e-4fcf-436f-b6a4-350e4077df1c service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:15.914020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1079/2144] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:15.942457 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714c03e-4fcf-436f-b6a4-350e4077df1c service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:15.957338 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714c03e-4fcf-436f-b6a4-350e4077df1c service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:15.958125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ada56-ffaa-4362-a97a-c6321a9e20cf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:15.963742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:15.964404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.964699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.965254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] PATH_INFO: `/v3/projects/3c4e77e362704c7dbbe5f55b08251629` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.965482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797adcc2-65fd-4bca-9e4f-eeca6ff99906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:15.967391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4e77e362704c7dbbe5f55b08251629)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:15.968155 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797adcc2-65fd-4bca-9e4f-eeca6ff99906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:15.968654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797adcc2-65fd-4bca-9e4f-eeca6ff99906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:15.969771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797adcc2-65fd-4bca-9e4f-eeca6ff99906 admin admin] PATH_INFO: `/v3/users/0d9203ca1da6465986c30444e92e6eed` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:15.970606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:15.972103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797adcc2-65fd-4bca-9e4f-eeca6ff99906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9203ca1da6465986c30444e92e6eed)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:15.975087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797adcc2-65fd-4bca-9e4f-eeca6ff99906 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:16.026018 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:16.049084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4e77e362704c7dbbe5f55b08251629', 'request_id': 'req-786dca3f-95a1-49da-a1a7-c03d13e220f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.056007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-797adcc2-65fd-4bca-9e4f-eeca6ff99906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9203ca1da6465986c30444e92e6eed'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.057264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-797adcc2-65fd-4bca-9e4f-eeca6ff99906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9203ca1da6465986c30444e92e6eed'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.066118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4e77e362704c7dbbe5f55b08251629', 'request_id': 'req-786dca3f-95a1-49da-a1a7-c03d13e220f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.072243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-797adcc2-65fd-4bca-9e4f-eeca6ff99906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9203ca1da6465986c30444e92e6eed'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.081014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4e77e362704c7dbbe5f55b08251629', 'request_id': 'req-786dca3f-95a1-49da-a1a7-c03d13e220f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.090138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-797adcc2-65fd-4bca-9e4f-eeca6ff99906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9203ca1da6465986c30444e92e6eed'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.102809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-797adcc2-65fd-4bca-9e4f-eeca6ff99906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9203ca1da6465986c30444e92e6eed'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.110442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] The token cache is being invalidate because project 3c4e77e362704c7dbbe5f55b08251629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:16.110969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1080/2145] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:15 2022] DELETE /identity/v3/users/0d9203ca1da6465986c30444e92e6eed => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:16.111081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.111677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.113425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1066/2146] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:15 2022] DELETE /identity/v3/projects/3c4e77e362704c7dbbe5f55b08251629 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:16.115330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab9137-de52-4fcc-bf1a-d396a9d216c9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:16.115619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab9137-de52-4fcc-bf1a-d396a9d216c9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:16.116158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab9137-de52-4fcc-bf1a-d396a9d216c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:16.117519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:16.118997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:16.138383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-98ab9137-de52-4fcc-bf1a-d396a9d216c9 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:16.140626 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-98ab9137-de52-4fcc-bf1a-d396a9d216c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:16.155293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:16.206378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dca3f-95a1-49da-a1a7-c03d13e220f2 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:16.220414 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1081/2147] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:16.231836 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab9137-de52-4fcc-bf1a-d396a9d216c9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:16.236242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-98ab9137-de52-4fcc-bf1a-d396a9d216c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:16.273910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:16.274828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:16.275338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:16.275591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] PATH_INFO: `/v3/users/bf83cb991956450e94317da9b6211c6c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:16.276814 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf83cb991956450e94317da9b6211c6c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:16.278753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:16.298424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab9137-de52-4fcc-bf1a-d396a9d216c9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:16.302621 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9082a-673c-4ad1-9bfa-81b63cd0d24f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:16.303271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9082a-673c-4ad1-9bfa-81b63cd0d24f service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:16.303630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9082a-673c-4ad1-9bfa-81b63cd0d24f service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:16.303911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9082a-673c-4ad1-9bfa-81b63cd0d24f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:16.316266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9082a-673c-4ad1-9bfa-81b63cd0d24f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:16.317732 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9082a-673c-4ad1-9bfa-81b63cd0d24f service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:16.328575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1082/2148] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:01:16.347693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf83cb991956450e94317da9b6211c6c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.348598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf83cb991956450e94317da9b6211c6c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.361656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf83cb991956450e94317da9b6211c6c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.375900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:16.379701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf83cb991956450e94317da9b6211c6c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.391079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf83cb991956450e94317da9b6211c6c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.408275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1067/2149] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:16 2022] DELETE /identity/v3/users/bf83cb991956450e94317da9b6211c6c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:16.714299 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9082a-673c-4ad1-9bfa-81b63cd0d24f service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:16.715769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-11e9082a-673c-4ad1-9bfa-81b63cd0d24f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:16.738501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9082a-673c-4ad1-9bfa-81b63cd0d24f service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:16.793019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:16.794501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:16.844445 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:16.845113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:16.845396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:16.845894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] PATH_INFO: `/v3/projects/c2f2154fa14b433f9beefe6148f62038` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:16.848279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f2154fa14b433f9beefe6148f62038)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:16.849741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:16.884354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c798af2-f539-41d3-a45b-cd7e6da7f862 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:16.890866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:16.891562 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:16.891856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:16.892134 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] PATH_INFO: `/v3/projects/bd659182a7b7425e9dae36d3192f9b5d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:16.893437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd659182a7b7425e9dae36d3192f9b5d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:16.895521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:16.909086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f2154fa14b433f9beefe6148f62038', 'request_id': 'req-a28780c7-8d57-4803-9225-6cd9463eef8e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.914922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f2154fa14b433f9beefe6148f62038', 'request_id': 'req-a28780c7-8d57-4803-9225-6cd9463eef8e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.916855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:16.923020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f2154fa14b433f9beefe6148f62038', 'request_id': 'req-a28780c7-8d57-4803-9225-6cd9463eef8e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.937895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] The token cache is being invalidate because project c2f2154fa14b433f9beefe6148f62038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:16.938103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.938749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a28780c7-8d57-4803-9225-6cd9463eef8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.940418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1083/2150] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:16 2022] DELETE /identity/v3/projects/c2f2154fa14b433f9beefe6148f62038 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:16.945192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ddfec-9ff9-40c2-9ed9-97e238fa1544 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:16.945507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ddfec-9ff9-40c2-9ed9-97e238fa1544 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:16.945790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ddfec-9ff9-40c2-9ed9-97e238fa1544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:16.947135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd659182a7b7425e9dae36d3192f9b5d', 'request_id': 'req-8b200e0e-c882-4743-a260-9d7a65822d60'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.953411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd659182a7b7425e9dae36d3192f9b5d', 'request_id': 'req-8b200e0e-c882-4743-a260-9d7a65822d60'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.965030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd659182a7b7425e9dae36d3192f9b5d', 'request_id': 'req-8b200e0e-c882-4743-a260-9d7a65822d60'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.970279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-802ddfec-9ff9-40c2-9ed9-97e238fa1544 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:16.971721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-802ddfec-9ff9-40c2-9ed9-97e238fa1544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:16.981534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] The token cache is being invalidate because project bd659182a7b7425e9dae36d3192f9b5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:16.981830 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.982498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b200e0e-c882-4743-a260-9d7a65822d60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:16.984559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1068/2151] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:16 2022] DELETE /identity/v3/projects/bd659182a7b7425e9dae36d3192f9b5d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:01:16.989023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04108aa8-8a2b-4df2-a2b2-c2606d08c708 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:16.989314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04108aa8-8a2b-4df2-a2b2-c2606d08c708 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:16.989592 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04108aa8-8a2b-4df2-a2b2-c2606d08c708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.019166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-04108aa8-8a2b-4df2-a2b2-c2606d08c708 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:17.019166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-04108aa8-8a2b-4df2-a2b2-c2606d08c708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:17.044430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1084/2152] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:17.052515 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfeb77d-6351-46c1-8995-82f4faa6784e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.052939 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfeb77d-6351-46c1-8995-82f4faa6784e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.053230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfeb77d-6351-46c1-8995-82f4faa6784e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.074478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-abfeb77d-6351-46c1-8995-82f4faa6784e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:17.075941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-abfeb77d-6351-46c1-8995-82f4faa6784e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:17.079833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1069/2153] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:17.087706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1085/2154] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:17.088029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda6699-90b3-4bed-85bc-5cc65b5b8f3e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.088125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda6699-90b3-4bed-85bc-5cc65b5b8f3e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.088393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda6699-90b3-4bed-85bc-5cc65b5b8f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.090182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfeb77d-6351-46c1-8995-82f4faa6784e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.099169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfeb77d-6351-46c1-8995-82f4faa6784e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.104584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d397c8-df47-4147-84e6-413608ff929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.105160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d397c8-df47-4147-84e6-413608ff929a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.105535 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d397c8-df47-4147-84e6-413608ff929a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.105818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d397c8-df47-4147-84e6-413608ff929a admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.106516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d397c8-df47-4147-84e6-413608ff929a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:17.106848 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d397c8-df47-4147-84e6-413608ff929a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.108577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d397c8-df47-4147-84e6-413608ff929a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.111363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6bda6699-90b3-4bed-85bc-5cc65b5b8f3e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:17.112581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6bda6699-90b3-4bed-85bc-5cc65b5b8f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:17.121682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1070/2155] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:17.126731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda6699-90b3-4bed-85bc-5cc65b5b8f3e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.137210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda6699-90b3-4bed-85bc-5cc65b5b8f3e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.142366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1086/2156] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:01:17.147339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73339213-4f7f-4321-8552-81b192aa1eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.147908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d397c8-df47-4147-84e6-413608ff929a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.148040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73339213-4f7f-4321-8552-81b192aa1eef admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.148267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73339213-4f7f-4321-8552-81b192aa1eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.148648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73339213-4f7f-4321-8552-81b192aa1eef admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.149361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73339213-4f7f-4321-8552-81b192aa1eef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:17.149743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73339213-4f7f-4321-8552-81b192aa1eef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.152806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73339213-4f7f-4321-8552-81b192aa1eef admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.155435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:17.168360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d397c8-df47-4147-84e6-413608ff929a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.172433 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1071/2157] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:01:17.173392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73339213-4f7f-4321-8552-81b192aa1eef admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.184008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3af51a-88fe-4dad-a0c9-b632c74250f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.184715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3af51a-88fe-4dad-a0c9-b632c74250f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.185210 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3af51a-88fe-4dad-a0c9-b632c74250f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.185691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3af51a-88fe-4dad-a0c9-b632c74250f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.187069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3af51a-88fe-4dad-a0c9-b632c74250f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.187349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73339213-4f7f-4321-8552-81b192aa1eef admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.189122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3af51a-88fe-4dad-a0c9-b632c74250f0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.193122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e4b14-ed99-45a5-84b8-b41a0fb73dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.194494 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e4b14-ed99-45a5-84b8-b41a0fb73dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.194879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e4b14-ed99-45a5-84b8-b41a0fb73dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.195161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e4b14-ed99-45a5-84b8-b41a0fb73dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.195842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e4b14-ed99-45a5-84b8-b41a0fb73dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.198713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e4b14-ed99-45a5-84b8-b41a0fb73dc1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.241828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1087/2158] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:17 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:17.246420 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3af51a-88fe-4dad-a0c9-b632c74250f0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.253629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1072/2159] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:17 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:17.259260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e4b14-ed99-45a5-84b8-b41a0fb73dc1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.320462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3af51a-88fe-4dad-a0c9-b632c74250f0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.350659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e4b14-ed99-45a5-84b8-b41a0fb73dc1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.355513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f6b7b-f67c-47c2-8ed0-9766bc703e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.356129 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6b7b-f67c-47c2-8ed0-9766bc703e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.356415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6b7b-f67c-47c2-8ed0-9766bc703e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.356697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6b7b-f67c-47c2-8ed0-9766bc703e6f admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.357368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f6b7b-f67c-47c2-8ed0-9766bc703e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.360638 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f6b7b-f67c-47c2-8ed0-9766bc703e6f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.375863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:17.385623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1073/2160] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:17 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:17.387909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2864532-4105-430e-837a-f2391aba9589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.390073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2864532-4105-430e-837a-f2391aba9589 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.391016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2864532-4105-430e-837a-f2391aba9589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.391437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f6b7b-f67c-47c2-8ed0-9766bc703e6f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.391610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2864532-4105-430e-837a-f2391aba9589 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.392217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2864532-4105-430e-837a-f2391aba9589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.394758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2864532-4105-430e-837a-f2391aba9589 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.401428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f6b7b-f67c-47c2-8ed0-9766bc703e6f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.407694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f23fa-8edf-4682-a535-398a652e5a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.408271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f23fa-8edf-4682-a535-398a652e5a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.408564 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f23fa-8edf-4682-a535-398a652e5a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.408844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f23fa-8edf-4682-a535-398a652e5a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.409640 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f23fa-8edf-4682-a535-398a652e5a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.411371 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f23fa-8edf-4682-a535-398a652e5a11 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.419090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1088/2161] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:17 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:17.424444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2864532-4105-430e-837a-f2391aba9589 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.424705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1074/2162] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:17.428431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f23fa-8edf-4682-a535-398a652e5a11 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.437817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2864532-4105-430e-837a-f2391aba9589 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.443468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f23fa-8edf-4682-a535-398a652e5a11 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.444869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799b37d-0f45-4172-8ddd-aa93b921ac19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.445616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799b37d-0f45-4172-8ddd-aa93b921ac19 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.445902 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799b37d-0f45-4172-8ddd-aa93b921ac19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.446169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799b37d-0f45-4172-8ddd-aa93b921ac19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.446924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799b37d-0f45-4172-8ddd-aa93b921ac19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.448510 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799b37d-0f45-4172-8ddd-aa93b921ac19 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.448713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0d8da-debd-497a-abd8-d1a450d85f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.449377 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0d8da-debd-497a-abd8-d1a450d85f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.449667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0d8da-debd-497a-abd8-d1a450d85f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.450520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0d8da-debd-497a-abd8-d1a450d85f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.451255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0d8da-debd-497a-abd8-d1a450d85f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.453952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0d8da-debd-497a-abd8-d1a450d85f44 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.461295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1089/2163] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:17.466242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799b37d-0f45-4172-8ddd-aa93b921ac19 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.470431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1075/2164] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:01:17.474388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0d8da-debd-497a-abd8-d1a450d85f44 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.480022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799b37d-0f45-4172-8ddd-aa93b921ac19 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.484668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0d8da-debd-497a-abd8-d1a450d85f44 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.485712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c25ab-ff49-4ba9-9e5b-615393e5b9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.486318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c25ab-ff49-4ba9-9e5b-615393e5b9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.486608 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c25ab-ff49-4ba9-9e5b-615393e5b9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.486898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c25ab-ff49-4ba9-9e5b-615393e5b9f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.487586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c25ab-ff49-4ba9-9e5b-615393e5b9f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.489168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c25ab-ff49-4ba9-9e5b-615393e5b9f7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.489526 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be04ac0-474c-44ba-9cea-ff1e3beb0bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.490193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be04ac0-474c-44ba-9cea-ff1e3beb0bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.490497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be04ac0-474c-44ba-9cea-ff1e3beb0bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.490756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be04ac0-474c-44ba-9cea-ff1e3beb0bfd admin admin] PATH_INFO: `/v3/projects/754ec94fff954ea2bac43b74ffedf519/users/9829c238da1f441fb0b612b2933213c7/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.502129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1090/2165] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:17.506588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c25ab-ff49-4ba9-9e5b-615393e5b9f7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.507362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be04ac0-474c-44ba-9cea-ff1e3beb0bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=754ec94fff954ea2bac43b74ffedf519, user_id=9829c238da1f441fb0b612b2933213c7, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.510113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be04ac0-474c-44ba-9cea-ff1e3beb0bfd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.519227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c25ab-ff49-4ba9-9e5b-615393e5b9f7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.526272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1076/2166] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:17 2022] PUT /identity/v3/projects/754ec94fff954ea2bac43b74ffedf519/users/9829c238da1f441fb0b612b2933213c7/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:01:17.539845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdee1e-ced8-4917-913a-1a2377db1b2a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.540285 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdee1e-ced8-4917-913a-1a2377db1b2a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.540687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdee1e-ced8-4917-913a-1a2377db1b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.579915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-17cdee1e-ced8-4917-913a-1a2377db1b2a None None] MFA Rules not processed for user `9829c238da1f441fb0b612b2933213c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:17.581652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-17cdee1e-ced8-4917-913a-1a2377db1b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:17.601192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b17250-245a-400b-9d85-5b75fe6ac190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.601863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b17250-245a-400b-9d85-5b75fe6ac190 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.602265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b17250-245a-400b-9d85-5b75fe6ac190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.602556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b17250-245a-400b-9d85-5b75fe6ac190 admin admin] PATH_INFO: `/v3/projects/5f99b34632fb4b32abe66ed1b57d7652/users/e24cd3abd38f4571b395ff73cb9448cc/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.625190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b17250-245a-400b-9d85-5b75fe6ac190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f99b34632fb4b32abe66ed1b57d7652, user_id=e24cd3abd38f4571b395ff73cb9448cc, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.627909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b17250-245a-400b-9d85-5b75fe6ac190 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.645102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1091/2167] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:17 2022] PUT /identity/v3/projects/5f99b34632fb4b32abe66ed1b57d7652/users/e24cd3abd38f4571b395ff73cb9448cc/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:17.656751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49ccf6-50e6-429c-897b-25428a78c363 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.657169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49ccf6-50e6-429c-897b-25428a78c363 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.657520 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49ccf6-50e6-429c-897b-25428a78c363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.668267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1077/2168] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:17.681027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdee1e-ced8-4917-913a-1a2377db1b2a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.682568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-17cdee1e-ced8-4917-913a-1a2377db1b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:17.687979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6e49ccf6-50e6-429c-897b-25428a78c363 None None] MFA Rules not processed for user `e24cd3abd38f4571b395ff73cb9448cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:17.689197 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6e49ccf6-50e6-429c-897b-25428a78c363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:17.693584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:17.761003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1092/2169] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:17.767263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdee1e-ced8-4917-913a-1a2377db1b2a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.772182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49ccf6-50e6-429c-897b-25428a78c363 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:17.773569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6e49ccf6-50e6-429c-897b-25428a78c363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:17.816077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:17.822537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82845e33-6648-4327-b199-6b2fc2cc85ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8967641dd624dacb5ff12c8e22599bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.823219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845e33-6648-4327-b199-6b2fc2cc85ac service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.823643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845e33-6648-4327-b199-6b2fc2cc85ac service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.823819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845e33-6648-4327-b199-6b2fc2cc85ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.840396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49ccf6-50e6-429c-897b-25428a78c363 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:17.845203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d1fa1-cc25-49f9-a8ac-7d7b7bdaa0c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:17.845835 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d1fa1-cc25-49f9-a8ac-7d7b7bdaa0c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:17.846127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d1fa1-cc25-49f9-a8ac-7d7b7bdaa0c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:17.846402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d1fa1-cc25-49f9-a8ac-7d7b7bdaa0c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:17.855831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d1fa1-cc25-49f9-a8ac-7d7b7bdaa0c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.857485 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d1fa1-cc25-49f9-a8ac-7d7b7bdaa0c5 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.870470 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1093/2170] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:01:17.891618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82845e33-6648-4327-b199-6b2fc2cc85ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:17.893270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82845e33-6648-4327-b199-6b2fc2cc85ac service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:17.903732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1078/2171] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:01:27.863218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=10 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:27.904828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=74 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:39.197055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d1fa1-cc25-49f9-a8ac-7d7b7bdaa0c5 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:39.275228 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d1fa1-cc25-49f9-a8ac-7d7b7bdaa0c5 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:39.281352 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:39.282748 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:39.283073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:39.283358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] PATH_INFO: `/v3/users/e24cd3abd38f4571b395ff73cb9448cc` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:39.284874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e24cd3abd38f4571b395ff73cb9448cc)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:39.287058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:39.366148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e24cd3abd38f4571b395ff73cb9448cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:39.367037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e24cd3abd38f4571b395ff73cb9448cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:39.382295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e24cd3abd38f4571b395ff73cb9448cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:39.390974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e24cd3abd38f4571b395ff73cb9448cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:39.399878 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e24cd3abd38f4571b395ff73cb9448cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:39.406003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1094/2172] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:39 2022] DELETE /identity/v3/users/e24cd3abd38f4571b395ff73cb9448cc => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:39.411384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000462db-aec4-4271-88ef-f7b602a59148 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:39.412237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000462db-aec4-4271-88ef-f7b602a59148 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:39.412237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000462db-aec4-4271-88ef-f7b602a59148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:39.436663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-000462db-aec4-4271-88ef-f7b602a59148 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:39.438266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-000462db-aec4-4271-88ef-f7b602a59148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:39.526834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1079/2173] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:39.540238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:39.541863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:39.614668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18ca66-cce0-483a-8a66-465e9ebc480d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:39.618988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad429c8-6281-4d4f-adbf-08c11e3ac6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:39.619682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad429c8-6281-4d4f-adbf-08c11e3ac6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:39.620021 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad429c8-6281-4d4f-adbf-08c11e3ac6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:39.620367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad429c8-6281-4d4f-adbf-08c11e3ac6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:39.632586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad429c8-6281-4d4f-adbf-08c11e3ac6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:39.635387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad429c8-6281-4d4f-adbf-08c11e3ac6b3 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:39.658347 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1095/2174] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:01:39.998210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000462db-aec4-4271-88ef-f7b602a59148 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:39.999741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-000462db-aec4-4271-88ef-f7b602a59148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:40.011618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000462db-aec4-4271-88ef-f7b602a59148 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:40.017025 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:40.017680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.017962 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.018237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] PATH_INFO: `/v3/projects/5f99b34632fb4b32abe66ed1b57d7652` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.019444 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f99b34632fb4b32abe66ed1b57d7652)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:40.021363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:40.069165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f99b34632fb4b32abe66ed1b57d7652', 'request_id': 'req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:40.074460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f99b34632fb4b32abe66ed1b57d7652', 'request_id': 'req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:40.082260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f99b34632fb4b32abe66ed1b57d7652', 'request_id': 'req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:40.093262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] The token cache is being invalidate because project 5f99b34632fb4b32abe66ed1b57d7652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:40.093545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:40.094251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9f7871fe-00ac-4e12-b3eb-a4f99970c07a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:40.096459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1080/2175] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:39 2022] DELETE /identity/v3/projects/5f99b34632fb4b32abe66ed1b57d7652 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:40.101397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f863a-8084-4056-9d66-5ef6bc906dc5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.101757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f863a-8084-4056-9d66-5ef6bc906dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.102044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f863a-8084-4056-9d66-5ef6bc906dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.121624 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c90f863a-8084-4056-9d66-5ef6bc906dc5 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:40.122867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c90f863a-8084-4056-9d66-5ef6bc906dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:40.140581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:40.185034 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1096/2176] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:40.193071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc71c3-c16d-4017-8d5e-33e287bd1f47 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.193357 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc71c3-c16d-4017-8d5e-33e287bd1f47 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.193659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc71c3-c16d-4017-8d5e-33e287bd1f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.214503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2ebc71c3-c16d-4017-8d5e-33e287bd1f47 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:40.215810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2ebc71c3-c16d-4017-8d5e-33e287bd1f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:40.224966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1081/2177] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:40.229617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f863a-8084-4056-9d66-5ef6bc906dc5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:40.240544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f863a-8084-4056-9d66-5ef6bc906dc5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:40.245263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d11e-fda6-4b29-9509-01399ac56228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:40.245891 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d11e-fda6-4b29-9509-01399ac56228 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.246179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d11e-fda6-4b29-9509-01399ac56228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.246524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d11e-fda6-4b29-9509-01399ac56228 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.247281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400d11e-fda6-4b29-9509-01399ac56228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:40.247596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400d11e-fda6-4b29-9509-01399ac56228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:40.249245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400d11e-fda6-4b29-9509-01399ac56228 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:40.261561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1097/2178] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:01:40.265955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc71c3-c16d-4017-8d5e-33e287bd1f47 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:40.275333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc71c3-c16d-4017-8d5e-33e287bd1f47 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:40.280059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1c2f8-fa17-480d-aab2-d5de7c1d155b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:40.280666 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1c2f8-fa17-480d-aab2-d5de7c1d155b admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.280953 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1c2f8-fa17-480d-aab2-d5de7c1d155b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.281233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1c2f8-fa17-480d-aab2-d5de7c1d155b admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.282422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1c2f8-fa17-480d-aab2-d5de7c1d155b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:40.284351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1c2f8-fa17-480d-aab2-d5de7c1d155b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:40.315019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1082/2179] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:40 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:40.319901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d11e-fda6-4b29-9509-01399ac56228 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:40.330156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d11e-fda6-4b29-9509-01399ac56228 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:40.393948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5662325-7965-4a07-a294-a8ce8ceb4d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:40.394802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662325-7965-4a07-a294-a8ce8ceb4d10 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.395261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662325-7965-4a07-a294-a8ce8ceb4d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.395539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662325-7965-4a07-a294-a8ce8ceb4d10 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.396209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5662325-7965-4a07-a294-a8ce8ceb4d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:40.398222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5662325-7965-4a07-a294-a8ce8ceb4d10 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:40.421147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1098/2180] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:40 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:40.426161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1c2f8-fa17-480d-aab2-d5de7c1d155b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:40.436622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1c2f8-fa17-480d-aab2-d5de7c1d155b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:40.441787 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987a2be-dc74-4fd5-a09f-10bc8974898d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:40.442479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987a2be-dc74-4fd5-a09f-10bc8974898d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.442773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987a2be-dc74-4fd5-a09f-10bc8974898d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.443120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987a2be-dc74-4fd5-a09f-10bc8974898d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.443866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987a2be-dc74-4fd5-a09f-10bc8974898d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:40.445521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987a2be-dc74-4fd5-a09f-10bc8974898d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:40.456251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1083/2181] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:40.460303 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5662325-7965-4a07-a294-a8ce8ceb4d10 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:40.471884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5662325-7965-4a07-a294-a8ce8ceb4d10 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:40.476990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee25058-082c-45d7-b36b-5126f0fb4b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:40.477508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee25058-082c-45d7-b36b-5126f0fb4b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.477792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee25058-082c-45d7-b36b-5126f0fb4b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.478064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee25058-082c-45d7-b36b-5126f0fb4b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.478761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee25058-082c-45d7-b36b-5126f0fb4b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:40.480362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee25058-082c-45d7-b36b-5126f0fb4b9f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:40.492968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1099/2182] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:40.497657 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987a2be-dc74-4fd5-a09f-10bc8974898d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:40.508475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987a2be-dc74-4fd5-a09f-10bc8974898d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:40.513831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6be32b-1506-43ab-88d0-305d889ef9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:40.514499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6be32b-1506-43ab-88d0-305d889ef9a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.514776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6be32b-1506-43ab-88d0-305d889ef9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.515086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6be32b-1506-43ab-88d0-305d889ef9a1 admin admin] PATH_INFO: `/v3/projects/58930d9271ed4cc38f26c4d450654eb9/users/d18838f2963e46a5ab793000385e8975/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.536628 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6be32b-1506-43ab-88d0-305d889ef9a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58930d9271ed4cc38f26c4d450654eb9, user_id=d18838f2963e46a5ab793000385e8975, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:40.539706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6be32b-1506-43ab-88d0-305d889ef9a1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:40.560276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1084/2183] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:40 2022] PUT /identity/v3/projects/58930d9271ed4cc38f26c4d450654eb9/users/d18838f2963e46a5ab793000385e8975/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:01:40.573180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9ec8e-10e1-4abe-89ab-80654945b59d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.573724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9ec8e-10e1-4abe-89ab-80654945b59d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.574485 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9ec8e-10e1-4abe-89ab-80654945b59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.610936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d3a9ec8e-10e1-4abe-89ab-80654945b59d None None] MFA Rules not processed for user `d18838f2963e46a5ab793000385e8975`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:40.612417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d3a9ec8e-10e1-4abe-89ab-80654945b59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:40.688569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1100/2184] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:40.692879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6be32b-1506-43ab-88d0-305d889ef9a1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:40.715373 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:40.763276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6be32b-1506-43ab-88d0-305d889ef9a1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:40.773354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e7d01-4096-4769-94b1-5f5571ea5058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:40.774670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e7d01-4096-4769-94b1-5f5571ea5058 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.775214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e7d01-4096-4769-94b1-5f5571ea5058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.775495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e7d01-4096-4769-94b1-5f5571ea5058 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.776537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e7d01-4096-4769-94b1-5f5571ea5058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:40.779057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e7d01-4096-4769-94b1-5f5571ea5058 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:40.818968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1085/2185] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:40 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:40.820159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9ec8e-10e1-4abe-89ab-80654945b59d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:40.833594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9ec8e-10e1-4abe-89ab-80654945b59d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:40.848260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:40.900061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a361552-461a-456f-8fc2-48914c6cbf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:40.900756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a361552-461a-456f-8fc2-48914c6cbf60 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.901040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a361552-461a-456f-8fc2-48914c6cbf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.901316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a361552-461a-456f-8fc2-48914c6cbf60 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.901996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a361552-461a-456f-8fc2-48914c6cbf60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:40.903959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a361552-461a-456f-8fc2-48914c6cbf60 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:40.932467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1101/2186] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:40 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:40.936670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e7d01-4096-4769-94b1-5f5571ea5058 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:40.949383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e7d01-4096-4769-94b1-5f5571ea5058 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:40.954515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c689c80-17de-4d63-8a83-ce307281cd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:40.955221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c689c80-17de-4d63-8a83-ce307281cd1f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.955454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c689c80-17de-4d63-8a83-ce307281cd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.955730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c689c80-17de-4d63-8a83-ce307281cd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.956467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c689c80-17de-4d63-8a83-ce307281cd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:40.958064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c689c80-17de-4d63-8a83-ce307281cd1f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:40.974816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1086/2187] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:40.978620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a361552-461a-456f-8fc2-48914c6cbf60 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:40.989257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a361552-461a-456f-8fc2-48914c6cbf60 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:40.994386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05363bbe-d13d-4379-a8fe-665da0c05131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:40.995034 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05363bbe-d13d-4379-a8fe-665da0c05131 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:40.995318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05363bbe-d13d-4379-a8fe-665da0c05131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:40.995595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05363bbe-d13d-4379-a8fe-665da0c05131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:40.996297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05363bbe-d13d-4379-a8fe-665da0c05131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:40.997882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05363bbe-d13d-4379-a8fe-665da0c05131 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:41.008258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1102/2188] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:41.012289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c689c80-17de-4d63-8a83-ce307281cd1f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:41.022484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c689c80-17de-4d63-8a83-ce307281cd1f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:41.027095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459dfc2-e7bb-4cf9-9ff0-a8e7de88091a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:41.027753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459dfc2-e7bb-4cf9-9ff0-a8e7de88091a admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:41.028038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459dfc2-e7bb-4cf9-9ff0-a8e7de88091a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:41.028321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459dfc2-e7bb-4cf9-9ff0-a8e7de88091a admin admin] PATH_INFO: `/v3/projects/e359f515bdcd4f15a1e5933c7746fad4/users/cd895c61df0b462292ae2a20791fcb72/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:41.044658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459dfc2-e7bb-4cf9-9ff0-a8e7de88091a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e359f515bdcd4f15a1e5933c7746fad4, user_id=cd895c61df0b462292ae2a20791fcb72, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:41.047988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459dfc2-e7bb-4cf9-9ff0-a8e7de88091a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:41.060670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1087/2189] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:41 2022] PUT /identity/v3/projects/e359f515bdcd4f15a1e5933c7746fad4/users/cd895c61df0b462292ae2a20791fcb72/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:41.070614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92abf17-c8f7-4866-8a19-34e6902d1cc5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:41.070940 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92abf17-c8f7-4866-8a19-34e6902d1cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:41.071223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92abf17-c8f7-4866-8a19-34e6902d1cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:41.099114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d92abf17-c8f7-4866-8a19-34e6902d1cc5 None None] MFA Rules not processed for user `cd895c61df0b462292ae2a20791fcb72`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:41.100341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d92abf17-c8f7-4866-8a19-34e6902d1cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:41.165124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1103/2190] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:41.169628 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459dfc2-e7bb-4cf9-9ff0-a8e7de88091a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:41.229347 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459dfc2-e7bb-4cf9-9ff0-a8e7de88091a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:41.233922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485723a-864a-4005-8e7e-ed94a2f408ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:41.234518 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485723a-864a-4005-8e7e-ed94a2f408ca admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:41.235396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485723a-864a-4005-8e7e-ed94a2f408ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:41.235659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485723a-864a-4005-8e7e-ed94a2f408ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:41.236496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485723a-864a-4005-8e7e-ed94a2f408ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:41.238384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485723a-864a-4005-8e7e-ed94a2f408ca admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:41.263968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1088/2191] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:41 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:41.268467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92abf17-c8f7-4866-8a19-34e6902d1cc5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:41.278968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92abf17-c8f7-4866-8a19-34e6902d1cc5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:41.343442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44196c90-0841-4c0a-b4ca-014e080a1487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:41.344067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44196c90-0841-4c0a-b4ca-014e080a1487 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:41.345022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44196c90-0841-4c0a-b4ca-014e080a1487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:41.345306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44196c90-0841-4c0a-b4ca-014e080a1487 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:41.345982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44196c90-0841-4c0a-b4ca-014e080a1487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:41.348138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44196c90-0841-4c0a-b4ca-014e080a1487 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:41.375954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1104/2192] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:41 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:41.377173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485723a-864a-4005-8e7e-ed94a2f408ca admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:41.387343 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485723a-864a-4005-8e7e-ed94a2f408ca admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:41.392570 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacaea3-af57-4466-ac6e-2557f2ee11dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:41.393179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacaea3-af57-4466-ac6e-2557f2ee11dd admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:41.393463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacaea3-af57-4466-ac6e-2557f2ee11dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:41.393739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacaea3-af57-4466-ac6e-2557f2ee11dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:41.394420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacaea3-af57-4466-ac6e-2557f2ee11dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:41.396027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacaea3-af57-4466-ac6e-2557f2ee11dd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:41.418065 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1089/2193] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:41.419228 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44196c90-0841-4c0a-b4ca-014e080a1487 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:41.429841 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44196c90-0841-4c0a-b4ca-014e080a1487 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:41.435377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a9a0f-fef1-493e-b584-1ce15029aa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:41.436435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a9a0f-fef1-493e-b584-1ce15029aa0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:41.436790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a9a0f-fef1-493e-b584-1ce15029aa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:41.437138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a9a0f-fef1-493e-b584-1ce15029aa0c admin admin] PATH_INFO: `/v3/projects/fc315aab5edc4e87aa05fd1aa9ff825e/users/41a20717d31d4c7a9e2cc0bb79240e4c/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:41.451787 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:41.456313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a9a0f-fef1-493e-b584-1ce15029aa0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc315aab5edc4e87aa05fd1aa9ff825e, user_id=41a20717d31d4c7a9e2cc0bb79240e4c, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:41.459307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a9a0f-fef1-493e-b584-1ce15029aa0c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:41.475377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1105/2194] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:41 2022] PUT /identity/v3/projects/fc315aab5edc4e87aa05fd1aa9ff825e/users/41a20717d31d4c7a9e2cc0bb79240e4c/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:41.479177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacaea3-af57-4466-ac6e-2557f2ee11dd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:41.490326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacaea3-af57-4466-ac6e-2557f2ee11dd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:41.536671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:41.552045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358ef3d-3a37-4275-b947-bca3eeb41084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:41.552812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358ef3d-3a37-4275-b947-bca3eeb41084 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:41.553169 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358ef3d-3a37-4275-b947-bca3eeb41084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:41.553545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358ef3d-3a37-4275-b947-bca3eeb41084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:41.554354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358ef3d-3a37-4275-b947-bca3eeb41084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:41.556108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358ef3d-3a37-4275-b947-bca3eeb41084 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:41.567078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1090/2195] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:41.571129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a9a0f-fef1-493e-b584-1ce15029aa0c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:41.583333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a9a0f-fef1-493e-b584-1ce15029aa0c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:41.588087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07afb2-2889-4a9f-a925-5cf42a53d11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:41.588851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07afb2-2889-4a9f-a925-5cf42a53d11f admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:41.589231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07afb2-2889-4a9f-a925-5cf42a53d11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:41.589569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07afb2-2889-4a9f-a925-5cf42a53d11f admin admin] PATH_INFO: `/v3/projects/fc315aab5edc4e87aa05fd1aa9ff825e/users/41a20717d31d4c7a9e2cc0bb79240e4c/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:41.591646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07afb2-2889-4a9f-a925-5cf42a53d11f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc315aab5edc4e87aa05fd1aa9ff825e, user_id=41a20717d31d4c7a9e2cc0bb79240e4c, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:41.594422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07afb2-2889-4a9f-a925-5cf42a53d11f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:41.605519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1106/2196] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:41 2022] PUT /identity/v3/projects/fc315aab5edc4e87aa05fd1aa9ff825e/users/41a20717d31d4c7a9e2cc0bb79240e4c/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:41.616070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07501b7-6e3b-41bb-a002-2d310fc8390b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:41.616367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07501b7-6e3b-41bb-a002-2d310fc8390b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:41.616706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07501b7-6e3b-41bb-a002-2d310fc8390b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:41.645394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f07501b7-6e3b-41bb-a002-2d310fc8390b None None] MFA Rules not processed for user `41a20717d31d4c7a9e2cc0bb79240e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:41.646621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f07501b7-6e3b-41bb-a002-2d310fc8390b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:41.718678 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1091/2197] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:41.730499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07afb2-2889-4a9f-a925-5cf42a53d11f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:41.732279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3a07afb2-2889-4a9f-a925-5cf42a53d11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:41.795592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07afb2-2889-4a9f-a925-5cf42a53d11f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:41.800583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f26c70-4402-4f4b-bd75-c2daf867ca5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:41.801207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f26c70-4402-4f4b-bd75-c2daf867ca5d service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:41.801516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f26c70-4402-4f4b-bd75-c2daf867ca5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:41.801789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f26c70-4402-4f4b-bd75-c2daf867ca5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:41.873620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f26c70-4402-4f4b-bd75-c2daf867ca5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:41.875113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f26c70-4402-4f4b-bd75-c2daf867ca5d service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:41.885663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1107/2198] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:46.607258 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07501b7-6e3b-41bb-a002-2d310fc8390b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:46.617171 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07501b7-6e3b-41bb-a002-2d310fc8390b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:46.620739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c011c2-ee18-405f-924d-f2d3affdf18a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:46.621350 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c011c2-ee18-405f-924d-f2d3affdf18a service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:46.621629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c011c2-ee18-405f-924d-f2d3affdf18a service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:46.621900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c011c2-ee18-405f-924d-f2d3affdf18a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:46.684904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c011c2-ee18-405f-924d-f2d3affdf18a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:46.686361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c011c2-ee18-405f-924d-f2d3affdf18a service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:46.696121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1092/2199] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:46.794419 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d3da2-60fa-43b4-831b-286ed0d26788 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:46.794961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d3da2-60fa-43b4-831b-286ed0d26788 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:46.795316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d3da2-60fa-43b4-831b-286ed0d26788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:46.815731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1b2d3da2-60fa-43b4-831b-286ed0d26788 None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:46.817394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1b2d3da2-60fa-43b4-831b-286ed0d26788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:46.879560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1108/2200] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 04:01:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:46.933561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c011c2-ee18-405f-924d-f2d3affdf18a service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:46.935616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-68c011c2-ee18-405f-924d-f2d3affdf18a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:47.032107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c011c2-ee18-405f-924d-f2d3affdf18a service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:47.040385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce116f-cca1-4e20-b111-0063a95b88e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:47.041024 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce116f-cca1-4e20-b111-0063a95b88e4 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:47.041311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce116f-cca1-4e20-b111-0063a95b88e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:47.041592 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce116f-cca1-4e20-b111-0063a95b88e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:47.048061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:47.052495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce116f-cca1-4e20-b111-0063a95b88e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:47.054650 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce116f-cca1-4e20-b111-0063a95b88e4 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:47.091760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1093/2201] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:01:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:01:47.640495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d3da2-60fa-43b4-831b-286ed0d26788 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:47.642731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1b2d3da2-60fa-43b4-831b-286ed0d26788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:47.658341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d3da2-60fa-43b4-831b-286ed0d26788 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:47.666874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3c593-c185-4c8d-b436-05c110cef326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:47.667825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3c593-c185-4c8d-b436-05c110cef326 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:47.668230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3c593-c185-4c8d-b436-05c110cef326 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:47.668515 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3c593-c185-4c8d-b436-05c110cef326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:47.682838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3c593-c185-4c8d-b436-05c110cef326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:47.684694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3c593-c185-4c8d-b436-05c110cef326 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:47.712080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1109/2202] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:49.239805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce116f-cca1-4e20-b111-0063a95b88e4 service swift] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:49.241474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e6ce116f-cca1-4e20-b111-0063a95b88e4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:49.334145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce116f-cca1-4e20-b111-0063a95b88e4 service swift] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:49.339837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:49.340538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:49.340911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:49.341260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] PATH_INFO: `/v3/users/9829c238da1f441fb0b612b2933213c7` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:49.342825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9829c238da1f441fb0b612b2933213c7)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:49.345735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:49.435288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9829c238da1f441fb0b612b2933213c7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:49.436248 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9829c238da1f441fb0b612b2933213c7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:49.461176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9829c238da1f441fb0b612b2933213c7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:49.480882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9829c238da1f441fb0b612b2933213c7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:49.490419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9829c238da1f441fb0b612b2933213c7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:49.497188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1094/2203] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:49 2022] DELETE /identity/v3/users/9829c238da1f441fb0b612b2933213c7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:49.502834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4b4b8-11e8-461c-a78e-881cb00cd018 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:49.503751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4b4b8-11e8-461c-a78e-881cb00cd018 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:49.504111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4b4b8-11e8-461c-a78e-881cb00cd018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:49.525011 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6af4b4b8-11e8-461c-a78e-881cb00cd018 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:49.526723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6af4b4b8-11e8-461c-a78e-881cb00cd018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:49.567524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:49.630105 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1110/2204] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:49.637473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:49.639398 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:49.735516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77356f0-7b59-49a6-9e9c-059ba25fff5d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:49.740444 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df04ae-cd23-4583-972a-c5b193977319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:49.741144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df04ae-cd23-4583-972a-c5b193977319 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:49.741607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df04ae-cd23-4583-972a-c5b193977319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:49.741898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df04ae-cd23-4583-972a-c5b193977319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:49.754024 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df04ae-cd23-4583-972a-c5b193977319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:49.755646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df04ae-cd23-4583-972a-c5b193977319 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:49.766370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1095/2205] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:01:50.121017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4b4b8-11e8-461c-a78e-881cb00cd018 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:50.122499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6af4b4b8-11e8-461c-a78e-881cb00cd018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:50.139750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4b4b8-11e8-461c-a78e-881cb00cd018 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:50.162374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:50.163546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:50.164824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:50.165700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] PATH_INFO: `/v3/projects/754ec94fff954ea2bac43b74ffedf519` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:50.169046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=754ec94fff954ea2bac43b74ffedf519)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:50.175215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:50.246728 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '754ec94fff954ea2bac43b74ffedf519', 'request_id': 'req-4bb341bd-1929-4fc7-8001-4e9934a9c541'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:50.254217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '754ec94fff954ea2bac43b74ffedf519', 'request_id': 'req-4bb341bd-1929-4fc7-8001-4e9934a9c541'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:50.264093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '754ec94fff954ea2bac43b74ffedf519', 'request_id': 'req-4bb341bd-1929-4fc7-8001-4e9934a9c541'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:50.286382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] The token cache is being invalidate because project 754ec94fff954ea2bac43b74ffedf519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:50.286685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:50.287373 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4bb341bd-1929-4fc7-8001-4e9934a9c541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:50.289823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1111/2206] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:50 2022] DELETE /identity/v3/projects/754ec94fff954ea2bac43b74ffedf519 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:50.296240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb8145-1adc-4f62-a1b0-6982dbc88070 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:50.296782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb8145-1adc-4f62-a1b0-6982dbc88070 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:50.297433 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb8145-1adc-4f62-a1b0-6982dbc88070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:50.332524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f1cb8145-1adc-4f62-a1b0-6982dbc88070 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:50.334403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f1cb8145-1adc-4f62-a1b0-6982dbc88070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:50.348718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:50.463428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1096/2207] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:50.473384 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4864fda-47d3-4261-bfe4-584a25e7cf40 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:50.473681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4864fda-47d3-4261-bfe4-584a25e7cf40 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:50.475316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4864fda-47d3-4261-bfe4-584a25e7cf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:50.512873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d4864fda-47d3-4261-bfe4-584a25e7cf40 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:50.513983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d4864fda-47d3-4261-bfe4-584a25e7cf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:50.523534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1112/2208] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:50.531459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb8145-1adc-4f62-a1b0-6982dbc88070 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:50.552911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb8145-1adc-4f62-a1b0-6982dbc88070 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:50.579676 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b2870-437a-4149-88e4-3283617a5872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:50.580488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b2870-437a-4149-88e4-3283617a5872 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:50.580782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b2870-437a-4149-88e4-3283617a5872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:50.581064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b2870-437a-4149-88e4-3283617a5872 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:50.581792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b2870-437a-4149-88e4-3283617a5872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:50.582104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b2870-437a-4149-88e4-3283617a5872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:50.584137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b2870-437a-4149-88e4-3283617a5872 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:50.605199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1097/2209] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:01:50.606911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4864fda-47d3-4261-bfe4-584a25e7cf40 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:50.617615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4864fda-47d3-4261-bfe4-584a25e7cf40 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:50.623553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9957b6-2749-43e6-b7eb-0de03bf1e830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:50.624300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9957b6-2749-43e6-b7eb-0de03bf1e830 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:50.624597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9957b6-2749-43e6-b7eb-0de03bf1e830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:50.624852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9957b6-2749-43e6-b7eb-0de03bf1e830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:50.627116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9957b6-2749-43e6-b7eb-0de03bf1e830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:50.630928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9957b6-2749-43e6-b7eb-0de03bf1e830 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:50.673781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1113/2210] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:50 2022] POST /identity/v3/projects => generated 363 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:50.675057 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b2870-437a-4149-88e4-3283617a5872 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:50.685277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b2870-437a-4149-88e4-3283617a5872 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:50.834557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f6fc1-070f-4277-8d7a-363e24d1394d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:50.835329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f6fc1-070f-4277-8d7a-363e24d1394d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:50.837365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f6fc1-070f-4277-8d7a-363e24d1394d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:50.837764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f6fc1-070f-4277-8d7a-363e24d1394d admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:50.840084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f6fc1-070f-4277-8d7a-363e24d1394d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:50.848571 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f6fc1-070f-4277-8d7a-363e24d1394d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:50.902683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1098/2211] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:50 2022] POST /identity/v3/users => generated 282 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:50.907024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9957b6-2749-43e6-b7eb-0de03bf1e830 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:50.921543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9957b6-2749-43e6-b7eb-0de03bf1e830 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:50.928883 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b519d19-4301-40ab-bb35-7421131f7d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:50.929802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519d19-4301-40ab-bb35-7421131f7d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:50.930162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519d19-4301-40ab-bb35-7421131f7d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:50.930668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519d19-4301-40ab-bb35-7421131f7d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:50.931745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b519d19-4301-40ab-bb35-7421131f7d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:50.933856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b519d19-4301-40ab-bb35-7421131f7d02 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:50.955856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1114/2212] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:50.956650 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f6fc1-070f-4277-8d7a-363e24d1394d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:50.973656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f6fc1-070f-4277-8d7a-363e24d1394d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:50.987302 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9e01a-e96a-4c60-ad6b-a793a4ed47a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:50.987945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9e01a-e96a-4c60-ad6b-a793a4ed47a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:50.988241 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9e01a-e96a-4c60-ad6b-a793a4ed47a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:50.988766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9e01a-e96a-4c60-ad6b-a793a4ed47a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:50.989587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9e01a-e96a-4c60-ad6b-a793a4ed47a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:50.991474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9e01a-e96a-4c60-ad6b-a793a4ed47a7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:51.016770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1099/2213] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:01:51.034031 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b519d19-4301-40ab-bb35-7421131f7d02 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:51.046483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b519d19-4301-40ab-bb35-7421131f7d02 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:51.053438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a702a-2de4-4377-b9fb-e466138300ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:51.054156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a702a-2de4-4377-b9fb-e466138300ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:51.054514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a702a-2de4-4377-b9fb-e466138300ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:51.055009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a702a-2de4-4377-b9fb-e466138300ce admin admin] PATH_INFO: `/v3/projects/8975512516924334ae7f26fceed80516/users/df31492def72471dbb0e328c0f4c4d1e/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:51.073188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a702a-2de4-4377-b9fb-e466138300ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=8975512516924334ae7f26fceed80516, user_id=df31492def72471dbb0e328c0f4c4d1e, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:51.076345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a702a-2de4-4377-b9fb-e466138300ce admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:51.093692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1115/2214] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:51 2022] PUT /identity/v3/projects/8975512516924334ae7f26fceed80516/users/df31492def72471dbb0e328c0f4c4d1e/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:51.104930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720fded-f97b-4ae5-ac62-1885639f25ec None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:51.105387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720fded-f97b-4ae5-ac62-1885639f25ec None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:51.105750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720fded-f97b-4ae5-ac62-1885639f25ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:51.143795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0720fded-f97b-4ae5-ac62-1885639f25ec None None] MFA Rules not processed for user `df31492def72471dbb0e328c0f4c4d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:51.145223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0720fded-f97b-4ae5-ac62-1885639f25ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:51.263905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1100/2215] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:51.272958 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a702a-2de4-4377-b9fb-e466138300ce admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:51.275203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6d8a702a-2de4-4377-b9fb-e466138300ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:51.330174 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:51.343052 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a702a-2de4-4377-b9fb-e466138300ce admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:51.349285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d7025-6da1-4430-a939-c19dab623294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:51.349933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d7025-6da1-4430-a939-c19dab623294 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:51.350253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d7025-6da1-4430-a939-c19dab623294 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:51.350709 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d7025-6da1-4430-a939-c19dab623294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:51.363437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d7025-6da1-4430-a939-c19dab623294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:51.366244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d7025-6da1-4430-a939-c19dab623294 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:51.382919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1116/2216] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:52.170935 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720fded-f97b-4ae5-ac62-1885639f25ec None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:52.173670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0720fded-f97b-4ae5-ac62-1885639f25ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:52.188726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:52.290066 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720fded-f97b-4ae5-ac62-1885639f25ec None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:52.296098 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:52.297626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:52.298092 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:52.298580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] PATH_INFO: `/v3/users/d18838f2963e46a5ab793000385e8975` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:52.299902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d18838f2963e46a5ab793000385e8975)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:52.302070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:52.375886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd18838f2963e46a5ab793000385e8975'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:52.377397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd18838f2963e46a5ab793000385e8975'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:52.387997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd18838f2963e46a5ab793000385e8975'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:52.405698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd18838f2963e46a5ab793000385e8975'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:52.417598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd18838f2963e46a5ab793000385e8975'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:52.429011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1101/2217] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:52 2022] DELETE /identity/v3/users/d18838f2963e46a5ab793000385e8975 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:52.433280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db029f-a2bd-4fa8-b278-92581ee1b4c6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:52.433801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db029f-a2bd-4fa8-b278-92581ee1b4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:52.433913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db029f-a2bd-4fa8-b278-92581ee1b4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:52.456276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-92db029f-a2bd-4fa8-b278-92581ee1b4c6 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:52.457501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-92db029f-a2bd-4fa8-b278-92581ee1b4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:52.531195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1117/2218] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:52.538442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:52.540365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:52.610034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fab09-b852-42d6-9edc-6a26b01d1d78 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:52.614131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd8c62-3229-4872-944f-e1b4d1fc2e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:52.614895 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8c62-3229-4872-944f-e1b4d1fc2e84 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:52.615247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8c62-3229-4872-944f-e1b4d1fc2e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:52.615595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8c62-3229-4872-944f-e1b4d1fc2e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:52.626318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd8c62-3229-4872-944f-e1b4d1fc2e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:52.627785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd8c62-3229-4872-944f-e1b4d1fc2e84 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:52.639822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1102/2219] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:53.227583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db029f-a2bd-4fa8-b278-92581ee1b4c6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:53.229213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-92db029f-a2bd-4fa8-b278-92581ee1b4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:53.248688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db029f-a2bd-4fa8-b278-92581ee1b4c6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:53.272840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:53.273513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:53.273867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:53.274151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] PATH_INFO: `/v3/projects/58930d9271ed4cc38f26c4d450654eb9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:53.275422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58930d9271ed4cc38f26c4d450654eb9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:53.278254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:53.366259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58930d9271ed4cc38f26c4d450654eb9', 'request_id': 'req-56673592-64fc-484a-9b72-73f277e2d548'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:53.385065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58930d9271ed4cc38f26c4d450654eb9', 'request_id': 'req-56673592-64fc-484a-9b72-73f277e2d548'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:53.394185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58930d9271ed4cc38f26c4d450654eb9', 'request_id': 'req-56673592-64fc-484a-9b72-73f277e2d548'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:53.416191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] The token cache is being invalidate because project 58930d9271ed4cc38f26c4d450654eb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:53.416486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:53.417138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:53.419939 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1118/2220] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:53 2022] DELETE /identity/v3/projects/58930d9271ed4cc38f26c4d450654eb9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:53.424228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd8c62-3229-4872-944f-e1b4d1fc2e84 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:53.461442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd8c62-3229-4872-944f-e1b4d1fc2e84 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:53.547726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:53.623934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:53.625429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:53.626547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:53.626917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] PATH_INFO: `/v3/users/cd895c61df0b462292ae2a20791fcb72` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:53.631351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd895c61df0b462292ae2a20791fcb72)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:53.634343 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:53.725421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd895c61df0b462292ae2a20791fcb72'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:53.727078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd895c61df0b462292ae2a20791fcb72'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:53.739899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd895c61df0b462292ae2a20791fcb72'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:53.765041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd895c61df0b462292ae2a20791fcb72'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:53.792216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd895c61df0b462292ae2a20791fcb72'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:53.803176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1103/2221] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:53 2022] DELETE /identity/v3/users/cd895c61df0b462292ae2a20791fcb72 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:54.155007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:54.158081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:54.190329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56673592-64fc-484a-9b72-73f277e2d548 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:54.396727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:54.397521 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:54.397877 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:54.398247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] PATH_INFO: `/v3/projects/e359f515bdcd4f15a1e5933c7746fad4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:54.400064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e359f515bdcd4f15a1e5933c7746fad4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:54.402377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:54.426445 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:54.503962 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e359f515bdcd4f15a1e5933c7746fad4', 'request_id': 'req-b8d0510d-8477-4aa8-af43-98c495f8a263'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:54.520984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e359f515bdcd4f15a1e5933c7746fad4', 'request_id': 'req-b8d0510d-8477-4aa8-af43-98c495f8a263'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:54.552967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e359f515bdcd4f15a1e5933c7746fad4', 'request_id': 'req-b8d0510d-8477-4aa8-af43-98c495f8a263'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:54.600893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] The token cache is being invalidate because project e359f515bdcd4f15a1e5933c7746fad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:54.601640 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:54.608462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:54.610243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1119/2222] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:54 2022] DELETE /identity/v3/projects/e359f515bdcd4f15a1e5933c7746fad4 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:54.614395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:54.616114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:54.658622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beee975-aad9-4d68-af17-a1d28f4e5da8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:54.776860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7fc05a-66fe-4386-ac53-b7aacc979338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:54.777535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fc05a-66fe-4386-ac53-b7aacc979338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:54.777824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fc05a-66fe-4386-ac53-b7aacc979338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:54.778107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fc05a-66fe-4386-ac53-b7aacc979338 admin admin] PATH_INFO: `/v3/users/41a20717d31d4c7a9e2cc0bb79240e4c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:54.779537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7fc05a-66fe-4386-ac53-b7aacc979338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a20717d31d4c7a9e2cc0bb79240e4c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:54.783422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7fc05a-66fe-4386-ac53-b7aacc979338 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:54.839812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-eb7fc05a-66fe-4386-ac53-b7aacc979338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a20717d31d4c7a9e2cc0bb79240e4c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:54.840476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-eb7fc05a-66fe-4386-ac53-b7aacc979338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a20717d31d4c7a9e2cc0bb79240e4c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:54.853874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-eb7fc05a-66fe-4386-ac53-b7aacc979338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a20717d31d4c7a9e2cc0bb79240e4c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:54.855693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:54.880216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-eb7fc05a-66fe-4386-ac53-b7aacc979338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a20717d31d4c7a9e2cc0bb79240e4c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:54.891149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-eb7fc05a-66fe-4386-ac53-b7aacc979338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a20717d31d4c7a9e2cc0bb79240e4c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:54.904882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1104/2223] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:54 2022] DELETE /identity/v3/users/41a20717d31d4c7a9e2cc0bb79240e4c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:55.215570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:55.217047 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:55.231687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0510d-8477-4aa8-af43-98c495f8a263 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:55.290580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:55.291256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:55.291543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:55.291821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] PATH_INFO: `/v3/projects/fc315aab5edc4e87aa05fd1aa9ff825e` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:55.292973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc315aab5edc4e87aa05fd1aa9ff825e)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:55.294899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:55.340738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc315aab5edc4e87aa05fd1aa9ff825e', 'request_id': 'req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:55.346880 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc315aab5edc4e87aa05fd1aa9ff825e', 'request_id': 'req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:55.352917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc315aab5edc4e87aa05fd1aa9ff825e', 'request_id': 'req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:55.366123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] The token cache is being invalidate because project fc315aab5edc4e87aa05fd1aa9ff825e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:01:55.366410 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:55.367111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f1badc6e-a8a0-4c0c-914c-ffb2430afc20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:55.368605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1120/2224] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:01:55 2022] DELETE /identity/v3/projects/fc315aab5edc4e87aa05fd1aa9ff825e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:55.373471 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8208b3-1753-4057-bd65-29082dceb7da None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:55.373778 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8208b3-1753-4057-bd65-29082dceb7da None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:55.374046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8208b3-1753-4057-bd65-29082dceb7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:55.395273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fc8208b3-1753-4057-bd65-29082dceb7da None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:55.396593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fc8208b3-1753-4057-bd65-29082dceb7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:55.456923 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1105/2225] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:01:55.465702 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291ec89-5d6b-4934-9a66-b18a8719ef63 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:55.465987 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291ec89-5d6b-4934-9a66-b18a8719ef63 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:55.466261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291ec89-5d6b-4934-9a66-b18a8719ef63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:55.486863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1291ec89-5d6b-4934-9a66-b18a8719ef63 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:55.487999 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1291ec89-5d6b-4934-9a66-b18a8719ef63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:55.496118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1121/2226] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:55.500684 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8208b3-1753-4057-bd65-29082dceb7da None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:55.510290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8208b3-1753-4057-bd65-29082dceb7da None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:55.514820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6724a-c00e-4a25-b846-a07c89435fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:55.515964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6724a-c00e-4a25-b846-a07c89435fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:55.516306 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6724a-c00e-4a25-b846-a07c89435fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:55.516586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6724a-c00e-4a25-b846-a07c89435fb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:55.517322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6724a-c00e-4a25-b846-a07c89435fb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:01:55.517622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6724a-c00e-4a25-b846-a07c89435fb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:55.519229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6724a-c00e-4a25-b846-a07c89435fb5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:55.530997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1106/2227] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:01:55.540100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291ec89-5d6b-4934-9a66-b18a8719ef63 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:55.550192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291ec89-5d6b-4934-9a66-b18a8719ef63 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:55.558074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fa172-6853-4c09-a3ba-2e5207c06ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:55.558688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fa172-6853-4c09-a3ba-2e5207c06ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:55.558999 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fa172-6853-4c09-a3ba-2e5207c06ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:55.559276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fa172-6853-4c09-a3ba-2e5207c06ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:55.559942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667fa172-6853-4c09-a3ba-2e5207c06ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:55.561838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667fa172-6853-4c09-a3ba-2e5207c06ee1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:55.595541 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1122/2228] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:01:55 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:01:55.600238 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6724a-c00e-4a25-b846-a07c89435fb5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:55.611090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6724a-c00e-4a25-b846-a07c89435fb5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:55.670322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e16747-e296-45ed-a49a-38a55dd040ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:55.671001 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e16747-e296-45ed-a49a-38a55dd040ee admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:55.671240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e16747-e296-45ed-a49a-38a55dd040ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:55.671516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e16747-e296-45ed-a49a-38a55dd040ee admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:55.672694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e16747-e296-45ed-a49a-38a55dd040ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:55.674594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e16747-e296-45ed-a49a-38a55dd040ee admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:55.695543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1107/2229] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:01:55 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:01:55.699728 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fa172-6853-4c09-a3ba-2e5207c06ee1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:55.705297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:55.712058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fa172-6853-4c09-a3ba-2e5207c06ee1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:55.718101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecff2a-4516-4e93-97c8-f282f587c3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:55.719003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecff2a-4516-4e93-97c8-f282f587c3eb admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:55.719257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecff2a-4516-4e93-97c8-f282f587c3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:55.719531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecff2a-4516-4e93-97c8-f282f587c3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:55.720234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecff2a-4516-4e93-97c8-f282f587c3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:55.721832 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecff2a-4516-4e93-97c8-f282f587c3eb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:55.731937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1123/2230] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:01:55.736425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e16747-e296-45ed-a49a-38a55dd040ee admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:55.746735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e16747-e296-45ed-a49a-38a55dd040ee admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:55.751967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c623520-c31b-4a77-a0bb-1943eacaaa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:55.752585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c623520-c31b-4a77-a0bb-1943eacaaa39 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:55.752869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c623520-c31b-4a77-a0bb-1943eacaaa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:55.753152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c623520-c31b-4a77-a0bb-1943eacaaa39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:55.753854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c623520-c31b-4a77-a0bb-1943eacaaa39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:55.755524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c623520-c31b-4a77-a0bb-1943eacaaa39 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:55.767099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1108/2231] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:01:55.771410 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecff2a-4516-4e93-97c8-f282f587c3eb admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:55.782432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecff2a-4516-4e93-97c8-f282f587c3eb admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:55.787159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2661fa-9ef7-4973-970b-c36527d8bc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:55.788076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2661fa-9ef7-4973-970b-c36527d8bc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:55.788362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2661fa-9ef7-4973-970b-c36527d8bc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:55.788647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2661fa-9ef7-4973-970b-c36527d8bc10 admin admin] PATH_INFO: `/v3/projects/d0ee73fc76264af295cc4b70746a010b/users/a84b7c92d6c04266ace6b8dee69fbdf6/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:55.813075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2661fa-9ef7-4973-970b-c36527d8bc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ee73fc76264af295cc4b70746a010b, user_id=a84b7c92d6c04266ace6b8dee69fbdf6, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:55.815786 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2661fa-9ef7-4973-970b-c36527d8bc10 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:55.829502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1124/2232] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:01:55 2022] PUT /identity/v3/projects/d0ee73fc76264af295cc4b70746a010b/users/a84b7c92d6c04266ace6b8dee69fbdf6/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:55.841052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604bea1-c1fe-4ad5-9874-6d6ea0c93e75 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:55.841408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604bea1-c1fe-4ad5-9874-6d6ea0c93e75 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:55.841770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604bea1-c1fe-4ad5-9874-6d6ea0c93e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:55.870698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7604bea1-c1fe-4ad5-9874-6d6ea0c93e75 None None] MFA Rules not processed for user `a84b7c92d6c04266ace6b8dee69fbdf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:55.872074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7604bea1-c1fe-4ad5-9874-6d6ea0c93e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:55.900107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:55.941080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1109/2233] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:55.952476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2661fa-9ef7-4973-970b-c36527d8bc10 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:55.954432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7b2661fa-9ef7-4973-970b-c36527d8bc10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:56.015622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2661fa-9ef7-4973-970b-c36527d8bc10 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:56.019483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964f238-0fe7-4785-b94e-b42cc330033f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:56.020100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f238-0fe7-4785-b94e-b42cc330033f service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:56.020386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f238-0fe7-4785-b94e-b42cc330033f service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:56.020663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f238-0fe7-4785-b94e-b42cc330033f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:56.029604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964f238-0fe7-4785-b94e-b42cc330033f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:56.031635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964f238-0fe7-4785-b94e-b42cc330033f service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:56.040736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1125/2234] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:01:59.660245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604bea1-c1fe-4ad5-9874-6d6ea0c93e75 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:59.721851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604bea1-c1fe-4ad5-9874-6d6ea0c93e75 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:59.726368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:59.727036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:59.727323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:59.727604 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] PATH_INFO: `/v3/users/a84b7c92d6c04266ace6b8dee69fbdf6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:59.728767 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a84b7c92d6c04266ace6b8dee69fbdf6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:59.730675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:59.776705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a84b7c92d6c04266ace6b8dee69fbdf6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:59.777290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a84b7c92d6c04266ace6b8dee69fbdf6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:59.784983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a84b7c92d6c04266ace6b8dee69fbdf6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:59.798435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a84b7c92d6c04266ace6b8dee69fbdf6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:59.805508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a84b7c92d6c04266ace6b8dee69fbdf6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:01:59.811645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1110/2235] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:01:59 2022] DELETE /identity/v3/users/a84b7c92d6c04266ace6b8dee69fbdf6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:01:59.816115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce2860-97ad-4bf8-b13f-81243e0c493c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:59.816590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce2860-97ad-4bf8-b13f-81243e0c493c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:59.816679 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce2860-97ad-4bf8-b13f-81243e0c493c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:59.836330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d8ce2860-97ad-4bf8-b13f-81243e0c493c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:01:59.837565 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d8ce2860-97ad-4bf8-b13f-81243e0c493c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:59.897033 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1126/2236] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:01:59.906877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:01:59.908317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:01:59.961574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:01:59.967624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae4e91-85b2-433a-b49a-b2160e40adb7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:01:59.972354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9e43a-5548-4eb3-a6ad-e5ef253dd544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:01:59.972972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9e43a-5548-4eb3-a6ad-e5ef253dd544 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:01:59.973261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9e43a-5548-4eb3-a6ad-e5ef253dd544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:01:59.973538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9e43a-5548-4eb3-a6ad-e5ef253dd544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:01:59.982412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9e43a-5548-4eb3-a6ad-e5ef253dd544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:01:59.983879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9e43a-5548-4eb3-a6ad-e5ef253dd544 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:01:59.993283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1111/2237] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:02:00.271604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce2860-97ad-4bf8-b13f-81243e0c493c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:00.273083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d8ce2860-97ad-4bf8-b13f-81243e0c493c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:00.286417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce2860-97ad-4bf8-b13f-81243e0c493c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:00.291324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:00.291954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:00.292347 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:00.292674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] PATH_INFO: `/v3/projects/d0ee73fc76264af295cc4b70746a010b` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:00.294071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ee73fc76264af295cc4b70746a010b)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:00.296121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:00.346036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ee73fc76264af295cc4b70746a010b', 'request_id': 'req-ebf309b3-bbe8-432e-9a9b-c9c32295af51'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:00.351586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ee73fc76264af295cc4b70746a010b', 'request_id': 'req-ebf309b3-bbe8-432e-9a9b-c9c32295af51'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:00.357729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ee73fc76264af295cc4b70746a010b', 'request_id': 'req-ebf309b3-bbe8-432e-9a9b-c9c32295af51'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:00.361181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:00.374119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] The token cache is being invalidate because project d0ee73fc76264af295cc4b70746a010b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:02:00.374519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:00.375315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ebf309b3-bbe8-432e-9a9b-c9c32295af51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:00.377751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1127/2238] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:02:00 2022] DELETE /identity/v3/projects/d0ee73fc76264af295cc4b70746a010b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:00.382592 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b8051-7b0f-4e06-9b71-4cd8ed2c4435 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:00.382913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b8051-7b0f-4e06-9b71-4cd8ed2c4435 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:00.383192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b8051-7b0f-4e06-9b71-4cd8ed2c4435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:00.403981 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4b5b8051-7b0f-4e06-9b71-4cd8ed2c4435 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:00.405353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4b5b8051-7b0f-4e06-9b71-4cd8ed2c4435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:00.465723 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1112/2239] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:02:00.474324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4b03d-7415-4fad-837d-cc4d5e084d1a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:00.474610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4b03d-7415-4fad-837d-cc4d5e084d1a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:00.474939 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4b03d-7415-4fad-837d-cc4d5e084d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:00.494755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-abe4b03d-7415-4fad-837d-cc4d5e084d1a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:00.496037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-abe4b03d-7415-4fad-837d-cc4d5e084d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:00.504761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1128/2240] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:00.508961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b8051-7b0f-4e06-9b71-4cd8ed2c4435 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:00.519374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b8051-7b0f-4e06-9b71-4cd8ed2c4435 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:00.524175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb031746-6e70-4364-9721-5e0ba7f99127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:00.524791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb031746-6e70-4364-9721-5e0ba7f99127 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:00.525078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb031746-6e70-4364-9721-5e0ba7f99127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:00.525359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb031746-6e70-4364-9721-5e0ba7f99127 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:00.526066 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb031746-6e70-4364-9721-5e0ba7f99127 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:02:00.526377 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb031746-6e70-4364-9721-5e0ba7f99127 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:00.528017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb031746-6e70-4364-9721-5e0ba7f99127 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:00.546471 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1113/2241] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:02:00.552701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4b03d-7415-4fad-837d-cc4d5e084d1a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:00.562120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4b03d-7415-4fad-837d-cc4d5e084d1a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:00.567788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9c04a-79fa-445b-a567-ae5a45c0099d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:00.568401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9c04a-79fa-445b-a567-ae5a45c0099d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:00.568818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9c04a-79fa-445b-a567-ae5a45c0099d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:00.569095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9c04a-79fa-445b-a567-ae5a45c0099d admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:00.569782 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9c04a-79fa-445b-a567-ae5a45c0099d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:00.571725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9c04a-79fa-445b-a567-ae5a45c0099d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:00.603187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1129/2242] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:02:00 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:02:00.604728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb031746-6e70-4364-9721-5e0ba7f99127 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:00.615898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb031746-6e70-4364-9721-5e0ba7f99127 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:00.680484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06171a3-a8de-4a36-a917-16053acfa085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:00.681160 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06171a3-a8de-4a36-a917-16053acfa085 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:00.681483 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06171a3-a8de-4a36-a917-16053acfa085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:00.681650 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06171a3-a8de-4a36-a917-16053acfa085 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:00.682334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06171a3-a8de-4a36-a917-16053acfa085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:00.684306 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06171a3-a8de-4a36-a917-16053acfa085 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:00.710925 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1114/2243] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:02:00 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:02:00.712172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9c04a-79fa-445b-a567-ae5a45c0099d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:00.723284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9c04a-79fa-445b-a567-ae5a45c0099d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:00.727951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbbf4b-92d6-49e0-9f16-9658a09f973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:00.728556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbbf4b-92d6-49e0-9f16-9658a09f973e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:00.728837 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbbf4b-92d6-49e0-9f16-9658a09f973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:00.729110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbbf4b-92d6-49e0-9f16-9658a09f973e admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:00.729808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dbbf4b-92d6-49e0-9f16-9658a09f973e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:00.731427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dbbf4b-92d6-49e0-9f16-9658a09f973e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:00.742221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1130/2244] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:02:00.745974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06171a3-a8de-4a36-a917-16053acfa085 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:00.757051 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06171a3-a8de-4a36-a917-16053acfa085 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:00.761660 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63e6ac-a546-4337-b558-3f016cca38fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:00.762267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63e6ac-a546-4337-b558-3f016cca38fa admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:00.762541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63e6ac-a546-4337-b558-3f016cca38fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:00.762848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63e6ac-a546-4337-b558-3f016cca38fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:00.763546 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63e6ac-a546-4337-b558-3f016cca38fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:00.765128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63e6ac-a546-4337-b558-3f016cca38fa admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:00.776939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1115/2245] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:02:00.780949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbbf4b-92d6-49e0-9f16-9658a09f973e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:00.791273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbbf4b-92d6-49e0-9f16-9658a09f973e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:00.795904 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583e3b1-6271-42ac-b6fd-02fb129be0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:00.796552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583e3b1-6271-42ac-b6fd-02fb129be0fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:00.796836 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583e3b1-6271-42ac-b6fd-02fb129be0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:00.797108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583e3b1-6271-42ac-b6fd-02fb129be0fd admin admin] PATH_INFO: `/v3/projects/38c83f50b239485f8bfcd414e420f538/users/7328233d36b94f48b3fa643a5e3f7381/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:00.811996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583e3b1-6271-42ac-b6fd-02fb129be0fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c83f50b239485f8bfcd414e420f538, user_id=7328233d36b94f48b3fa643a5e3f7381, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:00.815263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583e3b1-6271-42ac-b6fd-02fb129be0fd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:00.828082 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1131/2246] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:02:00 2022] PUT /identity/v3/projects/38c83f50b239485f8bfcd414e420f538/users/7328233d36b94f48b3fa643a5e3f7381/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:00.900478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc782ea-761e-4880-9c3d-42b7840da44c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:00.900930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc782ea-761e-4880-9c3d-42b7840da44c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:00.901284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc782ea-761e-4880-9c3d-42b7840da44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:00.936620 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2bc782ea-761e-4880-9c3d-42b7840da44c None None] MFA Rules not processed for user `7328233d36b94f48b3fa643a5e3f7381`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:00.937924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2bc782ea-761e-4880-9c3d-42b7840da44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:00.992936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:01.042867 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1116/2247] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:01.050902 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583e3b1-6271-42ac-b6fd-02fb129be0fd admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:01.052862 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8583e3b1-6271-42ac-b6fd-02fb129be0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:01.119137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583e3b1-6271-42ac-b6fd-02fb129be0fd admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:01.123170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13898f6-bc3c-4016-9648-badb119357f4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8967641dd624dacb5ff12c8e22599bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:01.124004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13898f6-bc3c-4016-9648-badb119357f4 service glance] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:01.124288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13898f6-bc3c-4016-9648-badb119357f4 service glance] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:01.124567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13898f6-bc3c-4016-9648-badb119357f4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:01.134559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13898f6-bc3c-4016-9648-badb119357f4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:01.136057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13898f6-bc3c-4016-9648-badb119357f4 service glance] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:01.145766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1132/2248] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:02:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:02:05.849425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5bff6-7b8d-439d-b14a-a000ffdbcfd3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:05.850008 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5bff6-7b8d-439d-b14a-a000ffdbcfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:05.850245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5bff6-7b8d-439d-b14a-a000ffdbcfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:05.869726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d1d5bff6-7b8d-439d-b14a-a000ffdbcfd3 None None] MFA Rules not processed for user `c3308761e9d2495e95e127bbc7b9c5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:05.870979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d1d5bff6-7b8d-439d-b14a-a000ffdbcfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:05.926569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1117/2249] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:11.141231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=1,keystone:SELECT=55,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:15.919204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=32 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:19.636678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073607e-df3a-4dcb-91cd-769382b7ead3 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:19.637275 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073607e-df3a-4dcb-91cd-769382b7ead3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:19.637275 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073607e-df3a-4dcb-91cd-769382b7ead3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:19.657568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2073607e-df3a-4dcb-91cd-769382b7ead3 None None] MFA Rules not processed for user `72dbbd19e4454bee84fe5bc797497bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:19.658726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2073607e-df3a-4dcb-91cd-769382b7ead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:19.722694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1133/2250] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:25.169523 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9f0a3-5efe-484c-8417-b8f64b97a9e2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:25.170111 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9f0a3-5efe-484c-8417-b8f64b97a9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:25.170111 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9f0a3-5efe-484c-8417-b8f64b97a9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:25.189598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-dbd9f0a3-5efe-484c-8417-b8f64b97a9e2 None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:25.191421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dbd9f0a3-5efe-484c-8417-b8f64b97a9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:25.254293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1118/2251] 10.222.0.46 () {58 vars in 967 bytes} [Thu Jul 28 04:02:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:25.268614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073607e-df3a-4dcb-91cd-769382b7ead3 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:25.270178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2073607e-df3a-4dcb-91cd-769382b7ead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:25.357498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073607e-df3a-4dcb-91cd-769382b7ead3 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:25.363430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c03b7-f9ba-4c15-9963-a4a8a80a5fcc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:25.364070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c03b7-f9ba-4c15-9963-a4a8a80a5fcc service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:25.364357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c03b7-f9ba-4c15-9963-a4a8a80a5fcc service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:25.364638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c03b7-f9ba-4c15-9963-a4a8a80a5fcc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:25.374138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c03b7-f9ba-4c15-9963-a4a8a80a5fcc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:25.375700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c03b7-f9ba-4c15-9963-a4a8a80a5fcc service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:25.384848 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1134/2252] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:02:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:02:28.232681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9f0a3-5efe-484c-8417-b8f64b97a9e2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:28.241993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dbd9f0a3-5efe-484c-8417-b8f64b97a9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:28.319208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9f0a3-5efe-484c-8417-b8f64b97a9e2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:28.325759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8429e5-c12b-47fe-98c3-fd83792bdde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:28.326428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8429e5-c12b-47fe-98c3-fd83792bdde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:28.326923 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8429e5-c12b-47fe-98c3-fd83792bdde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:28.327777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8429e5-c12b-47fe-98c3-fd83792bdde6 admin admin] PATH_INFO: `/v3/users/820e385c27af49f08301be61fcde369b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:28.329012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8429e5-c12b-47fe-98c3-fd83792bdde6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=820e385c27af49f08301be61fcde369b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:28.333010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8429e5-c12b-47fe-98c3-fd83792bdde6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:28.386927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ca8429e5-c12b-47fe-98c3-fd83792bdde6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820e385c27af49f08301be61fcde369b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:28.387604 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ca8429e5-c12b-47fe-98c3-fd83792bdde6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820e385c27af49f08301be61fcde369b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:28.395531 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ca8429e5-c12b-47fe-98c3-fd83792bdde6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820e385c27af49f08301be61fcde369b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:28.414689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ca8429e5-c12b-47fe-98c3-fd83792bdde6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820e385c27af49f08301be61fcde369b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:28.423360 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ca8429e5-c12b-47fe-98c3-fd83792bdde6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820e385c27af49f08301be61fcde369b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:28.430560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1119/2253] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:02:28 2022] DELETE /identity/v3/users/820e385c27af49f08301be61fcde369b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:28.864030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c03b7-f9ba-4c15-9963-a4a8a80a5fcc service swift] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:28.883183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c03b7-f9ba-4c15-9963-a4a8a80a5fcc service swift] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:28.949844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:28.950532 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:28.950916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:28.951203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] PATH_INFO: `/v3/projects/9ed8ce4c8a3442deacf65acc0c05dad3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:28.952853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ed8ce4c8a3442deacf65acc0c05dad3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:28.954960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:28.996543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:29.009120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ed8ce4c8a3442deacf65acc0c05dad3', 'request_id': 'req-b2e583ba-b892-42d2-bbb0-27051200fde0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:29.019996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ed8ce4c8a3442deacf65acc0c05dad3', 'request_id': 'req-b2e583ba-b892-42d2-bbb0-27051200fde0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:29.026826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ed8ce4c8a3442deacf65acc0c05dad3', 'request_id': 'req-b2e583ba-b892-42d2-bbb0-27051200fde0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:29.041107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] The token cache is being invalidate because project 9ed8ce4c8a3442deacf65acc0c05dad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:02:29.041399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:29.042028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b2e583ba-b892-42d2-bbb0-27051200fde0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:29.043603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1135/2254] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:02:28 2022] DELETE /identity/v3/projects/9ed8ce4c8a3442deacf65acc0c05dad3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:29.048788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff08c5-9374-49b0-a865-e5b730e24cf1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:29.049131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff08c5-9374-49b0-a865-e5b730e24cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:29.049414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff08c5-9374-49b0-a865-e5b730e24cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:29.072464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bdff08c5-9374-49b0-a865-e5b730e24cf1 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:29.073699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bdff08c5-9374-49b0-a865-e5b730e24cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:29.096188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:29.151166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1120/2255] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:29.152656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065de015-0875-4d56-9b50-e2c7c480f64e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:29.152952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065de015-0875-4d56-9b50-e2c7c480f64e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:29.153234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065de015-0875-4d56-9b50-e2c7c480f64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:29.175472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-065de015-0875-4d56-9b50-e2c7c480f64e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:29.176486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-065de015-0875-4d56-9b50-e2c7c480f64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:29.188592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1136/2256] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:02:29.190734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff08c5-9374-49b0-a865-e5b730e24cf1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:29.203434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff08c5-9374-49b0-a865-e5b730e24cf1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:29.208014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328188b6-b81b-49d0-a3f0-a851bb8e0ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:29.208014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328188b6-b81b-49d0-a3f0-a851bb8e0ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:29.208809 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328188b6-b81b-49d0-a3f0-a851bb8e0ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:29.208809 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328188b6-b81b-49d0-a3f0-a851bb8e0ad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:29.209297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328188b6-b81b-49d0-a3f0-a851bb8e0ad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:02:29.209613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328188b6-b81b-49d0-a3f0-a851bb8e0ad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:29.211289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328188b6-b81b-49d0-a3f0-a851bb8e0ad7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:29.224481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1121/2257] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:02:29.229356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065de015-0875-4d56-9b50-e2c7c480f64e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:29.239239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065de015-0875-4d56-9b50-e2c7c480f64e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:29.244597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295d4f0-4a93-42bc-a31c-d1f221b68f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:29.245475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295d4f0-4a93-42bc-a31c-d1f221b68f17 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:29.245764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295d4f0-4a93-42bc-a31c-d1f221b68f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:29.246069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295d4f0-4a93-42bc-a31c-d1f221b68f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:29.246751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295d4f0-4a93-42bc-a31c-d1f221b68f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:29.248757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295d4f0-4a93-42bc-a31c-d1f221b68f17 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:29.282355 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1137/2258] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:02:29 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:02:29.286540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328188b6-b81b-49d0-a3f0-a851bb8e0ad7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:29.298278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328188b6-b81b-49d0-a3f0-a851bb8e0ad7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:29.382927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2697a8-bb92-4379-be21-cab597a5191c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:29.383815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2697a8-bb92-4379-be21-cab597a5191c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:29.384176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2697a8-bb92-4379-be21-cab597a5191c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:29.384566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2697a8-bb92-4379-be21-cab597a5191c admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:29.385365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2697a8-bb92-4379-be21-cab597a5191c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:29.387943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2697a8-bb92-4379-be21-cab597a5191c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:29.412965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1122/2259] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:02:29 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:02:29.417049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295d4f0-4a93-42bc-a31c-d1f221b68f17 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:29.428752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295d4f0-4a93-42bc-a31c-d1f221b68f17 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:29.433991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495183d-e034-4c8b-80f9-b8b9d6ce82f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:29.434912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495183d-e034-4c8b-80f9-b8b9d6ce82f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:29.435262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495183d-e034-4c8b-80f9-b8b9d6ce82f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:29.435606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495183d-e034-4c8b-80f9-b8b9d6ce82f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:29.436384 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495183d-e034-4c8b-80f9-b8b9d6ce82f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:29.438101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495183d-e034-4c8b-80f9-b8b9d6ce82f4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:29.454884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1138/2260] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:02:29.458477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2697a8-bb92-4379-be21-cab597a5191c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:29.471595 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2697a8-bb92-4379-be21-cab597a5191c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:29.476879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afdcda-304f-46a7-a1fe-b6d49db75c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:29.477753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afdcda-304f-46a7-a1fe-b6d49db75c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:29.478128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afdcda-304f-46a7-a1fe-b6d49db75c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:29.478469 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afdcda-304f-46a7-a1fe-b6d49db75c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:29.479295 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afdcda-304f-46a7-a1fe-b6d49db75c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:29.480936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afdcda-304f-46a7-a1fe-b6d49db75c8c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:29.498319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1123/2261] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:02:29.499665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495183d-e034-4c8b-80f9-b8b9d6ce82f4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:29.520049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495183d-e034-4c8b-80f9-b8b9d6ce82f4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:29.524985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc42f6-b391-4001-9afb-9d00cd9ba871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:29.525670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc42f6-b391-4001-9afb-9d00cd9ba871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:29.525950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc42f6-b391-4001-9afb-9d00cd9ba871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:29.526232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc42f6-b391-4001-9afb-9d00cd9ba871 admin admin] PATH_INFO: `/v3/projects/2f5569f8399547068bd1daa108181f6d/users/39a14661ebce47819f268f68697efda6/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:29.563559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc42f6-b391-4001-9afb-9d00cd9ba871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5569f8399547068bd1daa108181f6d, user_id=39a14661ebce47819f268f68697efda6, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:29.563966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc42f6-b391-4001-9afb-9d00cd9ba871 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:29.587715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1139/2262] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:02:29 2022] PUT /identity/v3/projects/2f5569f8399547068bd1daa108181f6d/users/39a14661ebce47819f268f68697efda6/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:29.598473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08d85f-c89e-4148-9292-0aa270241058 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:29.598773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08d85f-c89e-4148-9292-0aa270241058 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:29.599089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08d85f-c89e-4148-9292-0aa270241058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:29.634061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fa08d85f-c89e-4148-9292-0aa270241058 None None] MFA Rules not processed for user `39a14661ebce47819f268f68697efda6`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:29.635432 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fa08d85f-c89e-4148-9292-0aa270241058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:29.710366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1124/2263] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:29.722734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc42f6-b391-4001-9afb-9d00cd9ba871 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:29.724466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-74fc42f6-b391-4001-9afb-9d00cd9ba871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:29.781293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc42f6-b391-4001-9afb-9d00cd9ba871 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:29.785660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e1431-a88e-4414-8a19-84cc18b6c978 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8967641dd624dacb5ff12c8e22599bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:29.786266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e1431-a88e-4414-8a19-84cc18b6c978 service glance] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:29.786544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e1431-a88e-4414-8a19-84cc18b6c978 service glance] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:29.786838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e1431-a88e-4414-8a19-84cc18b6c978 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:29.795647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e1431-a88e-4414-8a19-84cc18b6c978 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:29.797080 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e1431-a88e-4414-8a19-84cc18b6c978 service glance] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:29.807120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1140/2264] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:02:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:02:31.636167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92947c-3139-4d9b-a4a5-d492d901bd5f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:31.636695 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92947c-3139-4d9b-a4a5-d492d901bd5f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:31.636808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92947c-3139-4d9b-a4a5-d492d901bd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:31.661740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fc92947c-3139-4d9b-a4a5-d492d901bd5f None None] MFA Rules not processed for user `518ad5eb820e44aa9689215e6038a521`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:31.663313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fc92947c-3139-4d9b-a4a5-d492d901bd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:31.681737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:31.736965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1125/2265] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:33.802552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e1431-a88e-4414-8a19-84cc18b6c978 service glance] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:33.804069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-da5e1431-a88e-4414-8a19-84cc18b6c978 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:33.885056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e1431-a88e-4414-8a19-84cc18b6c978 service glance] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:33.894894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfacd5-6e58-45e2-a18a-389cd3fcaec2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:33.895581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfacd5-6e58-45e2-a18a-389cd3fcaec2 service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:33.896222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfacd5-6e58-45e2-a18a-389cd3fcaec2 service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:33.896496 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfacd5-6e58-45e2-a18a-389cd3fcaec2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:33.898823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dccfacd5-6e58-45e2-a18a-389cd3fcaec2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:33.907585 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:33.921752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfacd5-6e58-45e2-a18a-389cd3fcaec2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:33.923423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfacd5-6e58-45e2-a18a-389cd3fcaec2 service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:33.936029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1141/2266] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:02:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:02:41.725309 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=16 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:43.930875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:46.645551 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92947c-3139-4d9b-a4a5-d492d901bd5f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:46.647113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fc92947c-3139-4d9b-a4a5-d492d901bd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:46.727602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92947c-3139-4d9b-a4a5-d492d901bd5f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:46.732409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:46.733059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:46.733344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:46.733622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] PATH_INFO: `/v3/users/7328233d36b94f48b3fa643a5e3f7381` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:46.734822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7328233d36b94f48b3fa643a5e3f7381)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:46.737790 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:46.796311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7328233d36b94f48b3fa643a5e3f7381'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:46.797235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7328233d36b94f48b3fa643a5e3f7381'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:46.819680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7328233d36b94f48b3fa643a5e3f7381'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:46.835012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7328233d36b94f48b3fa643a5e3f7381'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:46.842370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7328233d36b94f48b3fa643a5e3f7381'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:46.850280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1126/2267] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:02:46 2022] DELETE /identity/v3/users/7328233d36b94f48b3fa643a5e3f7381 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:46.855458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480f288-76d9-4f38-858f-0ec6952991d5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:46.855930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480f288-76d9-4f38-858f-0ec6952991d5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:46.856420 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480f288-76d9-4f38-858f-0ec6952991d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:46.876953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8480f288-76d9-4f38-858f-0ec6952991d5 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:46.878284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8480f288-76d9-4f38-858f-0ec6952991d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:46.947953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1142/2268] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:02:46.957601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:46.959071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:47.020498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b534f-7347-46e7-91a9-f566a422b73c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:47.024297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6f564-adad-4dad-b67d-531ca9992427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:47.024925 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6f564-adad-4dad-b67d-531ca9992427 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:47.025216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6f564-adad-4dad-b67d-531ca9992427 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:47.025499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6f564-adad-4dad-b67d-531ca9992427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:47.036271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6f564-adad-4dad-b67d-531ca9992427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:47.039303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6f564-adad-4dad-b67d-531ca9992427 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:47.054084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1127/2269] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:02:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:02:47.511176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480f288-76d9-4f38-858f-0ec6952991d5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:47.513534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8480f288-76d9-4f38-858f-0ec6952991d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:47.535743 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480f288-76d9-4f38-858f-0ec6952991d5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:47.547638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:47.548349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:47.548912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:47.548912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] PATH_INFO: `/v3/projects/38c83f50b239485f8bfcd414e420f538` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:47.550143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c83f50b239485f8bfcd414e420f538)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:47.552106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:47.686362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c83f50b239485f8bfcd414e420f538', 'request_id': 'req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:47.711000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c83f50b239485f8bfcd414e420f538', 'request_id': 'req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:47.718666 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c83f50b239485f8bfcd414e420f538', 'request_id': 'req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:47.738257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] The token cache is being invalidate because project 38c83f50b239485f8bfcd414e420f538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:02:47.738558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:47.739225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a7dfcbc9-d2f2-4600-abbd-dadec20a0ad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:47.740957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1143/2270] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:02:47 2022] DELETE /identity/v3/projects/38c83f50b239485f8bfcd414e420f538 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:52.744542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6f564-adad-4dad-b67d-531ca9992427 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:52.745953 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-19f6f564-adad-4dad-b67d-531ca9992427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:52.758769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6f564-adad-4dad-b67d-531ca9992427 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:52.777236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:52.818371 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:52.819047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:52.819336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:52.819609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] PATH_INFO: `/v3/users/df31492def72471dbb0e328c0f4c4d1e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:52.820784 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] RBAC: Authorizing `identity:delete_user(user_id=df31492def72471dbb0e328c0f4c4d1e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:52.822708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:52.868357 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df31492def72471dbb0e328c0f4c4d1e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:52.869018 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df31492def72471dbb0e328c0f4c4d1e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:52.878305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df31492def72471dbb0e328c0f4c4d1e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:52.890935 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df31492def72471dbb0e328c0f4c4d1e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:52.899556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df31492def72471dbb0e328c0f4c4d1e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:52.905605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1128/2271] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:02:52 2022] DELETE /identity/v3/users/df31492def72471dbb0e328c0f4c4d1e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:52.910228 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43be237-d817-4f85-aeb6-09f71e9e4b7f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:52.910712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43be237-d817-4f85-aeb6-09f71e9e4b7f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:52.910838 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43be237-d817-4f85-aeb6-09f71e9e4b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:52.929857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e43be237-d817-4f85-aeb6-09f71e9e4b7f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:52.931084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e43be237-d817-4f85-aeb6-09f71e9e4b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:52.989662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1144/2272] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:52.999440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:53.000834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:53.058745 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400dc2e-54dc-4954-acc7-c3bfab888ccb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:53.062480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb77cd-846e-4514-829a-123329221d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:53.063137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb77cd-846e-4514-829a-123329221d14 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:53.063421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb77cd-846e-4514-829a-123329221d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:53.063697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb77cd-846e-4514-829a-123329221d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:53.073329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb77cd-846e-4514-829a-123329221d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:53.074752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb77cd-846e-4514-829a-123329221d14 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:53.084194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1129/2273] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:02:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:02:53.357969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43be237-d817-4f85-aeb6-09f71e9e4b7f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:53.359549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e43be237-d817-4f85-aeb6-09f71e9e4b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:53.371511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43be237-d817-4f85-aeb6-09f71e9e4b7f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:53.376180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:53.376888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:53.377380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:53.377668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] PATH_INFO: `/v3/projects/8975512516924334ae7f26fceed80516` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:53.378875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8975512516924334ae7f26fceed80516)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:53.380822 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:53.404106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:53.431770 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8975512516924334ae7f26fceed80516', 'request_id': 'req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:53.437236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8975512516924334ae7f26fceed80516', 'request_id': 'req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:53.443458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8975512516924334ae7f26fceed80516', 'request_id': 'req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:53.456355 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] The token cache is being invalidate because project 8975512516924334ae7f26fceed80516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:02:53.456642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:53.457272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-22f646ea-3a3d-4848-a0b9-ac76076fa0b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:53.458798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1145/2274] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:02:53 2022] DELETE /identity/v3/projects/8975512516924334ae7f26fceed80516 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:53.463638 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad656b4-2125-4c16-b2a7-0a98e3faf802 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:53.463933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad656b4-2125-4c16-b2a7-0a98e3faf802 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:53.464218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad656b4-2125-4c16-b2a7-0a98e3faf802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:53.484045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bad656b4-2125-4c16-b2a7-0a98e3faf802 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:53.485249 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bad656b4-2125-4c16-b2a7-0a98e3faf802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:53.526686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:53.548517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1130/2275] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:53.556501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9798a7-b118-41e7-8707-d2c9e119e10e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:53.556796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9798a7-b118-41e7-8707-d2c9e119e10e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:53.557075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9798a7-b118-41e7-8707-d2c9e119e10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:53.577345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cd9798a7-b118-41e7-8707-d2c9e119e10e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:53.578549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cd9798a7-b118-41e7-8707-d2c9e119e10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:53.587108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1146/2276] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:02:53.591485 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad656b4-2125-4c16-b2a7-0a98e3faf802 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:53.601547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad656b4-2125-4c16-b2a7-0a98e3faf802 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:53.606130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f1033-8e01-40a9-8932-108e3bf9a1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:53.606733 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f1033-8e01-40a9-8932-108e3bf9a1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:53.607042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f1033-8e01-40a9-8932-108e3bf9a1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:53.607313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f1033-8e01-40a9-8932-108e3bf9a1e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:53.608001 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f1033-8e01-40a9-8932-108e3bf9a1e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:02:53.608301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f1033-8e01-40a9-8932-108e3bf9a1e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:53.609908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f1033-8e01-40a9-8932-108e3bf9a1e3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:53.622465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1131/2277] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:02:53.627277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9798a7-b118-41e7-8707-d2c9e119e10e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:53.636652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9798a7-b118-41e7-8707-d2c9e119e10e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:53.641348 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4272e4-7675-40d6-9b53-37e8934ee863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:53.642581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4272e4-7675-40d6-9b53-37e8934ee863 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:53.642975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4272e4-7675-40d6-9b53-37e8934ee863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:53.643257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4272e4-7675-40d6-9b53-37e8934ee863 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:53.643918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4272e4-7675-40d6-9b53-37e8934ee863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:53.645936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4272e4-7675-40d6-9b53-37e8934ee863 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:53.675926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1147/2278] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:02:53 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:02:53.678184 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f1033-8e01-40a9-8932-108e3bf9a1e3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:53.689237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f1033-8e01-40a9-8932-108e3bf9a1e3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:53.744021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76832c54-606a-463f-b598-b207903c7c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:53.744627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76832c54-606a-463f-b598-b207903c7c65 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:53.745162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76832c54-606a-463f-b598-b207903c7c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:53.745162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76832c54-606a-463f-b598-b207903c7c65 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:53.745802 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76832c54-606a-463f-b598-b207903c7c65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:53.747748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76832c54-606a-463f-b598-b207903c7c65 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:53.770513 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1132/2279] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:02:53 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:02:53.774767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4272e4-7675-40d6-9b53-37e8934ee863 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:53.785084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4272e4-7675-40d6-9b53-37e8934ee863 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:53.789866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d9d58-4e40-4381-b13c-197a4bd874fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:53.790465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d9d58-4e40-4381-b13c-197a4bd874fd admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:53.790745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d9d58-4e40-4381-b13c-197a4bd874fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:53.791054 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d9d58-4e40-4381-b13c-197a4bd874fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:53.791892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d9d58-4e40-4381-b13c-197a4bd874fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:53.793591 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d9d58-4e40-4381-b13c-197a4bd874fd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:53.803378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1148/2280] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:02:53.807211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76832c54-606a-463f-b598-b207903c7c65 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:53.817628 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76832c54-606a-463f-b598-b207903c7c65 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:53.822444 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ca4c4-63ec-420c-86a9-03060efded49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:53.823084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ca4c4-63ec-420c-86a9-03060efded49 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:53.823361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ca4c4-63ec-420c-86a9-03060efded49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:53.823632 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ca4c4-63ec-420c-86a9-03060efded49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:53.824321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ca4c4-63ec-420c-86a9-03060efded49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:53.825917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ca4c4-63ec-420c-86a9-03060efded49 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:53.837005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1133/2281] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:02:53.840820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d9d58-4e40-4381-b13c-197a4bd874fd admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:53.850388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d9d58-4e40-4381-b13c-197a4bd874fd admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:53.854993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d8950-edcc-4cdb-a858-9593d6addd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:53.855634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d8950-edcc-4cdb-a858-9593d6addd1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:53.855915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d8950-edcc-4cdb-a858-9593d6addd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:53.856190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d8950-edcc-4cdb-a858-9593d6addd1f admin admin] PATH_INFO: `/v3/projects/bec1296604ee4201bffc66fc36504c8a/users/eec4df0b0b5840a78a1e20893f388ec9/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:53.871321 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d8950-edcc-4cdb-a858-9593d6addd1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec1296604ee4201bffc66fc36504c8a, user_id=eec4df0b0b5840a78a1e20893f388ec9, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:53.873964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d8950-edcc-4cdb-a858-9593d6addd1f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:53.886919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1149/2282] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:02:53 2022] PUT /identity/v3/projects/bec1296604ee4201bffc66fc36504c8a/users/eec4df0b0b5840a78a1e20893f388ec9/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:53.898717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd487c-3bdd-4f93-9c86-6bbc14915a09 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:53.899033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd487c-3bdd-4f93-9c86-6bbc14915a09 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:53.899310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd487c-3bdd-4f93-9c86-6bbc14915a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:53.927612 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-26fd487c-3bdd-4f93-9c86-6bbc14915a09 None None] MFA Rules not processed for user `eec4df0b0b5840a78a1e20893f388ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:53.929135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-26fd487c-3bdd-4f93-9c86-6bbc14915a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:53.993562 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1134/2283] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:54.004525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d8950-edcc-4cdb-a858-9593d6addd1f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:54.006279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3b8d8950-edcc-4cdb-a858-9593d6addd1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:54.062439 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d8950-edcc-4cdb-a858-9593d6addd1f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:54.066570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c81d88-4415-439c-b56f-f24032008416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:54.067209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c81d88-4415-439c-b56f-f24032008416 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:54.067491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c81d88-4415-439c-b56f-f24032008416 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:54.067766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c81d88-4415-439c-b56f-f24032008416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:54.075898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c81d88-4415-439c-b56f-f24032008416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:54.077315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c81d88-4415-439c-b56f-f24032008416 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:54.086448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1150/2284] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:02:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:02:54.111764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd487c-3bdd-4f93-9c86-6bbc14915a09 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:54.152621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:54.174613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd487c-3bdd-4f93-9c86-6bbc14915a09 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:54.179144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:54.179776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:54.180061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:54.180344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] PATH_INFO: `/v3/users/eec4df0b0b5840a78a1e20893f388ec9` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:54.181508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eec4df0b0b5840a78a1e20893f388ec9)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:54.183461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:54.228791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eec4df0b0b5840a78a1e20893f388ec9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:54.229460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eec4df0b0b5840a78a1e20893f388ec9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:54.236623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eec4df0b0b5840a78a1e20893f388ec9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:54.248648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eec4df0b0b5840a78a1e20893f388ec9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:54.256114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eec4df0b0b5840a78a1e20893f388ec9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:54.262009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1135/2285] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:02:54 2022] DELETE /identity/v3/users/eec4df0b0b5840a78a1e20893f388ec9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:54.266433 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f30330-4daa-4f8c-b28b-833902778059 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:54.266731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f30330-4daa-4f8c-b28b-833902778059 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:54.267553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f30330-4daa-4f8c-b28b-833902778059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:54.286490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-93f30330-4daa-4f8c-b28b-833902778059 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:54.287981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-93f30330-4daa-4f8c-b28b-833902778059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:54.328794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:54.348334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1151/2286] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:54.358030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:54.359464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:54.417311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd10822-f353-43e5-88d8-9bf157ab0d6d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:54.421194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c5dc3-7a1e-4642-87c7-4dc894ed96f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:54.421836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c5dc3-7a1e-4642-87c7-4dc894ed96f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:54.422119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c5dc3-7a1e-4642-87c7-4dc894ed96f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:54.422396 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c5dc3-7a1e-4642-87c7-4dc894ed96f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:54.431862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c5dc3-7a1e-4642-87c7-4dc894ed96f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:54.433289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c5dc3-7a1e-4642-87c7-4dc894ed96f8 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:54.443436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1136/2287] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:02:54.713305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f30330-4daa-4f8c-b28b-833902778059 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:54.715066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-93f30330-4daa-4f8c-b28b-833902778059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:54.727589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f30330-4daa-4f8c-b28b-833902778059 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:54.732375 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:54.732966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:54.733263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:54.733522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] PATH_INFO: `/v3/projects/bec1296604ee4201bffc66fc36504c8a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:54.734671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bec1296604ee4201bffc66fc36504c8a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:54.737304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:54.783501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bec1296604ee4201bffc66fc36504c8a', 'request_id': 'req-2f274419-7bbf-4005-a489-5d5d1c21f817'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:54.789092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bec1296604ee4201bffc66fc36504c8a', 'request_id': 'req-2f274419-7bbf-4005-a489-5d5d1c21f817'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:54.795176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bec1296604ee4201bffc66fc36504c8a', 'request_id': 'req-2f274419-7bbf-4005-a489-5d5d1c21f817'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:54.808195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] The token cache is being invalidate because project bec1296604ee4201bffc66fc36504c8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:02:54.808488 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:54.809114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2f274419-7bbf-4005-a489-5d5d1c21f817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:02:54.810645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1152/2288] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:02:54 2022] DELETE /identity/v3/projects/bec1296604ee4201bffc66fc36504c8a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:54.815760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8aa1c-e086-468a-ada1-dd250ffc16c3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:54.816238 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8aa1c-e086-468a-ada1-dd250ffc16c3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:54.816342 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8aa1c-e086-468a-ada1-dd250ffc16c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:54.835916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d6f8aa1c-e086-468a-ada1-dd250ffc16c3 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:54.837040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d6f8aa1c-e086-468a-ada1-dd250ffc16c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:54.896065 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1137/2289] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:54.904190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a8f40-40ba-4fb4-a7b8-ee824c86f0f7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:54.904479 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a8f40-40ba-4fb4-a7b8-ee824c86f0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:54.904756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a8f40-40ba-4fb4-a7b8-ee824c86f0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:54.925064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c09a8f40-40ba-4fb4-a7b8-ee824c86f0f7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:54.926270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c09a8f40-40ba-4fb4-a7b8-ee824c86f0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:54.934016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1153/2290] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:54.938190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8aa1c-e086-468a-ada1-dd250ffc16c3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:54.943440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:54.950004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8aa1c-e086-468a-ada1-dd250ffc16c3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:54.956251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbca56-ff57-4d8f-9a75-a90ea1b0e5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:54.956859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbca56-ff57-4d8f-9a75-a90ea1b0e5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:54.957147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbca56-ff57-4d8f-9a75-a90ea1b0e5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:54.957427 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbca56-ff57-4d8f-9a75-a90ea1b0e5d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:54.958128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbca56-ff57-4d8f-9a75-a90ea1b0e5d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:02:54.958437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbca56-ff57-4d8f-9a75-a90ea1b0e5d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:54.960104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbca56-ff57-4d8f-9a75-a90ea1b0e5d7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:54.972442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1138/2291] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:02:54.976622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a8f40-40ba-4fb4-a7b8-ee824c86f0f7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:54.986073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a8f40-40ba-4fb4-a7b8-ee824c86f0f7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:54.991065 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1a702-4bd0-45c2-8a62-3b1abb03efb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:54.991673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1a702-4bd0-45c2-8a62-3b1abb03efb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:54.991957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1a702-4bd0-45c2-8a62-3b1abb03efb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:54.992231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1a702-4bd0-45c2-8a62-3b1abb03efb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:54.992901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1a702-4bd0-45c2-8a62-3b1abb03efb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:54.994852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1a702-4bd0-45c2-8a62-3b1abb03efb4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:55.024144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1154/2292] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:02:54 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:02:55.035362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbca56-ff57-4d8f-9a75-a90ea1b0e5d7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:55.039474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbca56-ff57-4d8f-9a75-a90ea1b0e5d7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:55.096447 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee60aa-7f28-41e5-90f3-06672354aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:55.096987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee60aa-7f28-41e5-90f3-06672354aaef admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:55.097268 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee60aa-7f28-41e5-90f3-06672354aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:55.097538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee60aa-7f28-41e5-90f3-06672354aaef admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:55.098207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee60aa-7f28-41e5-90f3-06672354aaef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:55.100176 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee60aa-7f28-41e5-90f3-06672354aaef admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:55.120482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1139/2293] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:02:55 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:02:55.124645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1a702-4bd0-45c2-8a62-3b1abb03efb4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:55.135453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1a702-4bd0-45c2-8a62-3b1abb03efb4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:55.140077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f33c85-43fa-4ae9-b129-21fb6c6f727c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:55.140680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f33c85-43fa-4ae9-b129-21fb6c6f727c admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:55.140966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f33c85-43fa-4ae9-b129-21fb6c6f727c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:55.141249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f33c85-43fa-4ae9-b129-21fb6c6f727c admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:55.142050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f33c85-43fa-4ae9-b129-21fb6c6f727c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:55.143686 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f33c85-43fa-4ae9-b129-21fb6c6f727c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:55.153554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1155/2294] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:02:55.157223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee60aa-7f28-41e5-90f3-06672354aaef admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:55.167872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee60aa-7f28-41e5-90f3-06672354aaef admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:55.172487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87098f3-eae3-4ba8-be97-6e7374899355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:55.173659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87098f3-eae3-4ba8-be97-6e7374899355 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:55.173934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87098f3-eae3-4ba8-be97-6e7374899355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:55.174210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87098f3-eae3-4ba8-be97-6e7374899355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:55.174938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87098f3-eae3-4ba8-be97-6e7374899355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:55.176547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87098f3-eae3-4ba8-be97-6e7374899355 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:55.186805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1140/2295] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:02:55.190929 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f33c85-43fa-4ae9-b129-21fb6c6f727c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:55.201018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f33c85-43fa-4ae9-b129-21fb6c6f727c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:55.205724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2136f-1691-48a4-9b9d-535916bad9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:55.206371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2136f-1691-48a4-9b9d-535916bad9d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:55.206657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2136f-1691-48a4-9b9d-535916bad9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:55.206979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2136f-1691-48a4-9b9d-535916bad9d4 admin admin] PATH_INFO: `/v3/projects/338679f381ed4e9e898a59289b0f8178/users/352d60372ba249e38ea4aa2d62723f19/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:55.221789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2136f-1691-48a4-9b9d-535916bad9d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=338679f381ed4e9e898a59289b0f8178, user_id=352d60372ba249e38ea4aa2d62723f19, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:55.224480 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2136f-1691-48a4-9b9d-535916bad9d4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:55.237725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1156/2296] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:02:55 2022] PUT /identity/v3/projects/338679f381ed4e9e898a59289b0f8178/users/352d60372ba249e38ea4aa2d62723f19/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:55.242661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ad8fa-f904-4a3b-a0c8-8e6c449e2533 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:55.243004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ad8fa-f904-4a3b-a0c8-8e6c449e2533 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:55.243288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ad8fa-f904-4a3b-a0c8-8e6c449e2533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:55.262799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7b7ad8fa-f904-4a3b-a0c8-8e6c449e2533 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:55.264054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7b7ad8fa-f904-4a3b-a0c8-8e6c449e2533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:55.323859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1141/2297] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:55.334130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2136f-1691-48a4-9b9d-535916bad9d4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:55.335919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f7c2136f-1691-48a4-9b9d-535916bad9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:55.393079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2136f-1691-48a4-9b9d-535916bad9d4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:55.396662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50b31-5981-4215-b4e0-db76735c9165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:55.397277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50b31-5981-4215-b4e0-db76735c9165 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:55.397566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50b31-5981-4215-b4e0-db76735c9165 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:55.397846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50b31-5981-4215-b4e0-db76735c9165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:55.406764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee50b31-5981-4215-b4e0-db76735c9165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:55.408234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee50b31-5981-4215-b4e0-db76735c9165 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:55.417803 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1157/2298] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:02:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:02:59.453053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18ea22-aa91-48d8-91de-6d97ec97c2af None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:59.453594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18ea22-aa91-48d8-91de-6d97ec97c2af None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:59.453594 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18ea22-aa91-48d8-91de-6d97ec97c2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:59.483721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0e18ea22-aa91-48d8-91de-6d97ec97c2af None None] MFA Rules not processed for user `352d60372ba249e38ea4aa2d62723f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:02:59.484937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0e18ea22-aa91-48d8-91de-6d97ec97c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:02:59.558336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:59.562037 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1142/2299] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:02:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:02:59.566917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50b31-5981-4215-b4e0-db76735c9165 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:59.578553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50b31-5981-4215-b4e0-db76735c9165 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:59.584137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112c1ce-e465-4154-917e-b07fd5158e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:59.584756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112c1ce-e465-4154-917e-b07fd5158e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:59.585035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112c1ce-e465-4154-917e-b07fd5158e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:59.585309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112c1ce-e465-4154-917e-b07fd5158e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:59.585972 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112c1ce-e465-4154-917e-b07fd5158e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:59.588233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112c1ce-e465-4154-917e-b07fd5158e35 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:59.593994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:02:59.645996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1158/2300] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:02:59 2022] POST /identity/v3/projects => generated 379 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:02:59.650132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18ea22-aa91-48d8-91de-6d97ec97c2af None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:59.668463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18ea22-aa91-48d8-91de-6d97ec97c2af None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:59.732119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f80bb-dad7-43c8-a111-e8efd066167d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:59.732751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f80bb-dad7-43c8-a111-e8efd066167d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:59.733030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f80bb-dad7-43c8-a111-e8efd066167d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:59.733303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f80bb-dad7-43c8-a111-e8efd066167d admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:59.733987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f80bb-dad7-43c8-a111-e8efd066167d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:59.736166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f80bb-dad7-43c8-a111-e8efd066167d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:59.761574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1143/2301] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:02:59 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:02:59.765437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112c1ce-e465-4154-917e-b07fd5158e35 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:59.776950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112c1ce-e465-4154-917e-b07fd5158e35 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:59.781986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c229ac-ba41-4e0c-b311-ef2edddf902f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:59.783154 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c229ac-ba41-4e0c-b311-ef2edddf902f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:59.783613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c229ac-ba41-4e0c-b311-ef2edddf902f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:59.784045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c229ac-ba41-4e0c-b311-ef2edddf902f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:59.784755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c229ac-ba41-4e0c-b311-ef2edddf902f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:59.786355 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c229ac-ba41-4e0c-b311-ef2edddf902f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:59.797285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1159/2302] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:02:59.801693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f80bb-dad7-43c8-a111-e8efd066167d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:59.812338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f80bb-dad7-43c8-a111-e8efd066167d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:59.817941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b16545-e9bf-48c2-954c-a891a3b38451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:59.818663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b16545-e9bf-48c2-954c-a891a3b38451 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:59.818983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b16545-e9bf-48c2-954c-a891a3b38451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:59.819263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b16545-e9bf-48c2-954c-a891a3b38451 admin admin] PATH_INFO: `/v3/projects/e7aed6d258a0490a97f4ffc6de84193b/users/ac14f1b5d01741468a111a80c72e2763/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:59.837155 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b16545-e9bf-48c2-954c-a891a3b38451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7aed6d258a0490a97f4ffc6de84193b, user_id=ac14f1b5d01741468a111a80c72e2763, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:59.839872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b16545-e9bf-48c2-954c-a891a3b38451 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:59.853774 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1144/2303] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:02:59 2022] PUT /identity/v3/projects/e7aed6d258a0490a97f4ffc6de84193b/users/ac14f1b5d01741468a111a80c72e2763/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:02:59.857690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c229ac-ba41-4e0c-b311-ef2edddf902f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:59.868885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c229ac-ba41-4e0c-b311-ef2edddf902f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:59.929550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64848aa1-8496-4709-a95f-99bbf5d03af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:59.930184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64848aa1-8496-4709-a95f-99bbf5d03af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:59.930469 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64848aa1-8496-4709-a95f-99bbf5d03af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:59.930746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64848aa1-8496-4709-a95f-99bbf5d03af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:59.931599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64848aa1-8496-4709-a95f-99bbf5d03af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:59.933245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64848aa1-8496-4709-a95f-99bbf5d03af3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:59.944564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1160/2304] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:02:59.948703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b16545-e9bf-48c2-954c-a891a3b38451 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:02:59.959618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b16545-e9bf-48c2-954c-a891a3b38451 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:02:59.964409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82335a32-b76a-4236-8800-940a7eb9235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:02:59.965080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335a32-b76a-4236-8800-940a7eb9235a admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:02:59.965361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335a32-b76a-4236-8800-940a7eb9235a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:02:59.965636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335a32-b76a-4236-8800-940a7eb9235a admin admin] PATH_INFO: `/v3/projects/e7aed6d258a0490a97f4ffc6de84193b/users/ac14f1b5d01741468a111a80c72e2763/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:02:59.967635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82335a32-b76a-4236-8800-940a7eb9235a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7aed6d258a0490a97f4ffc6de84193b, user_id=ac14f1b5d01741468a111a80c72e2763, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:02:59.970354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82335a32-b76a-4236-8800-940a7eb9235a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:02:59.982151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1145/2305] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:02:59 2022] PUT /identity/v3/projects/e7aed6d258a0490a97f4ffc6de84193b/users/ac14f1b5d01741468a111a80c72e2763/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:03:04.406163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd437b-78c9-4cbe-bc1d-8f06acd2ee49 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:04.406662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd437b-78c9-4cbe-bc1d-8f06acd2ee49 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:04.406745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd437b-78c9-4cbe-bc1d-8f06acd2ee49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:04.436255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-dadd437b-78c9-4cbe-bc1d-8f06acd2ee49 None None] MFA Rules not processed for user `ac14f1b5d01741468a111a80c72e2763`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:04.437475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dadd437b-78c9-4cbe-bc1d-8f06acd2ee49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:04.509997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1161/2306] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:03:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:03:04.526842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82335a32-b76a-4236-8800-940a7eb9235a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:04.528669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-82335a32-b76a-4236-8800-940a7eb9235a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:04.607038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82335a32-b76a-4236-8800-940a7eb9235a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:04.613375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bf913-f6d8-4a45-8774-d354f9b60a54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:04.614003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bf913-f6d8-4a45-8774-d354f9b60a54 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:04.614287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bf913-f6d8-4a45-8774-d354f9b60a54 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:04.614567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bf913-f6d8-4a45-8774-d354f9b60a54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:04.661961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:03:04.675073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bf913-f6d8-4a45-8774-d354f9b60a54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:04.676718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bf913-f6d8-4a45-8774-d354f9b60a54 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:04.686012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1146/2307] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:03:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:03:14.506755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=73,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:03:14.681560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:03:26.965661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd437b-78c9-4cbe-bc1d-8f06acd2ee49 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:26.967138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dadd437b-78c9-4cbe-bc1d-8f06acd2ee49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:27.038692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd437b-78c9-4cbe-bc1d-8f06acd2ee49 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:27.043681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:27.044361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:27.044631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:27.045044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] PATH_INFO: `/v3/users/39a14661ebce47819f268f68697efda6` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:27.046270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a14661ebce47819f268f68697efda6)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:27.048294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:27.105503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a14661ebce47819f268f68697efda6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:27.106146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a14661ebce47819f268f68697efda6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:27.118230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a14661ebce47819f268f68697efda6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:27.125083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a14661ebce47819f268f68697efda6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:27.134684 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a14661ebce47819f268f68697efda6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:27.142195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1162/2308] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:03:26 2022] DELETE /identity/v3/users/39a14661ebce47819f268f68697efda6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:03:27.147394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd82eb-97fc-4c5d-a5e7-327816d9daea None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:27.147843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd82eb-97fc-4c5d-a5e7-327816d9daea None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:27.147991 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd82eb-97fc-4c5d-a5e7-327816d9daea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:27.172482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-82dd82eb-97fc-4c5d-a5e7-327816d9daea None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:27.174207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-82dd82eb-97fc-4c5d-a5e7-327816d9daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:27.241299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1147/2309] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:03:27.251111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:27.253181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:27.315950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b575f39-d333-433f-b412-16858b1635c5 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:27.319586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8e57a-89d7-434c-b089-93e0633f84e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:27.320210 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8e57a-89d7-434c-b089-93e0633f84e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:27.320495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8e57a-89d7-434c-b089-93e0633f84e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:27.320775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8e57a-89d7-434c-b089-93e0633f84e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:27.330823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8e57a-89d7-434c-b089-93e0633f84e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:27.331950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8e57a-89d7-434c-b089-93e0633f84e2 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:27.341986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1163/2310] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:03:27.608760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd82eb-97fc-4c5d-a5e7-327816d9daea None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:27.610343 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-82dd82eb-97fc-4c5d-a5e7-327816d9daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:27.632602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd82eb-97fc-4c5d-a5e7-327816d9daea None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:27.638257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:27.639145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:27.639768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:27.639888 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] PATH_INFO: `/v3/projects/2f5569f8399547068bd1daa108181f6d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:27.641363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f5569f8399547068bd1daa108181f6d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:27.643666 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:27.695689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f5569f8399547068bd1daa108181f6d', 'request_id': 'req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:27.701435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f5569f8399547068bd1daa108181f6d', 'request_id': 'req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:27.710347 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f5569f8399547068bd1daa108181f6d', 'request_id': 'req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:27.722605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] The token cache is being invalidate because project 2f5569f8399547068bd1daa108181f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:03:27.722955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:27.723621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-168c4f9d-f74b-433d-9b7b-0614e3a1fef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:27.725182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1148/2311] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:03:27 2022] DELETE /identity/v3/projects/2f5569f8399547068bd1daa108181f6d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:03:27.730077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceb8a8-e584-4464-ab0a-f494a57066ec None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:27.730369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceb8a8-e584-4464-ab0a-f494a57066ec None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:27.730651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceb8a8-e584-4464-ab0a-f494a57066ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:27.753930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3bceb8a8-e584-4464-ab0a-f494a57066ec None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:27.755256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3bceb8a8-e584-4464-ab0a-f494a57066ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:27.774307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:03:27.819109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1164/2312] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:03:27.828222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80c35f-989f-43e7-9987-4873fbaf539f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:27.828509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80c35f-989f-43e7-9987-4873fbaf539f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:27.828788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80c35f-989f-43e7-9987-4873fbaf539f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:27.850141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ac80c35f-989f-43e7-9987-4873fbaf539f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:27.851366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ac80c35f-989f-43e7-9987-4873fbaf539f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:27.859989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1149/2313] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:03:27.864109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bceb8a8-e584-4464-ab0a-f494a57066ec None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:27.873525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bceb8a8-e584-4464-ab0a-f494a57066ec None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:27.879152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa6076-ad66-4688-849e-8e267cf354a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:27.879758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa6076-ad66-4688-849e-8e267cf354a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:27.880038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa6076-ad66-4688-849e-8e267cf354a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:27.880313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa6076-ad66-4688-849e-8e267cf354a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:27.881007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa6076-ad66-4688-849e-8e267cf354a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:03:27.881309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa6076-ad66-4688-849e-8e267cf354a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:27.882927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa6076-ad66-4688-849e-8e267cf354a4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:27.896217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1165/2314] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:03:27.900651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80c35f-989f-43e7-9987-4873fbaf539f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:27.910079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80c35f-989f-43e7-9987-4873fbaf539f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:27.914990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef85fb-fecd-47d3-8302-28058a09c8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:27.915622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef85fb-fecd-47d3-8302-28058a09c8cf admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:27.915900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef85fb-fecd-47d3-8302-28058a09c8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:27.916173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef85fb-fecd-47d3-8302-28058a09c8cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:27.916852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef85fb-fecd-47d3-8302-28058a09c8cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:27.918760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef85fb-fecd-47d3-8302-28058a09c8cf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:27.950547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1150/2315] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:03:27 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:03:28.013639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa6076-ad66-4688-849e-8e267cf354a4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:28.024044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa6076-ad66-4688-849e-8e267cf354a4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:28.080082 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7382ae1-10ee-4534-9d97-967e299af018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:28.080708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7382ae1-10ee-4534-9d97-967e299af018 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:28.080992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7382ae1-10ee-4534-9d97-967e299af018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:28.081266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7382ae1-10ee-4534-9d97-967e299af018 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:28.081943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7382ae1-10ee-4534-9d97-967e299af018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:28.083887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7382ae1-10ee-4534-9d97-967e299af018 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:28.105210 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1166/2316] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:03:28 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:03:28.109317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef85fb-fecd-47d3-8302-28058a09c8cf admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:28.120327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef85fb-fecd-47d3-8302-28058a09c8cf admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:28.125104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f3382-b310-4876-878d-122cea646086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:28.125707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f3382-b310-4876-878d-122cea646086 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:28.125992 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f3382-b310-4876-878d-122cea646086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:28.126270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f3382-b310-4876-878d-122cea646086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:28.127088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f3382-b310-4876-878d-122cea646086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:28.128666 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f3382-b310-4876-878d-122cea646086 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:28.139381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1151/2317] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:03:28.143537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7382ae1-10ee-4534-9d97-967e299af018 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:28.153451 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7382ae1-10ee-4534-9d97-967e299af018 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:28.158561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ae5e2-c01a-44d3-b520-dd7d5a06ed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:28.159194 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ae5e2-c01a-44d3-b520-dd7d5a06ed4a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:28.159481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ae5e2-c01a-44d3-b520-dd7d5a06ed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:28.159758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ae5e2-c01a-44d3-b520-dd7d5a06ed4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:28.160454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ae5e2-c01a-44d3-b520-dd7d5a06ed4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:28.162044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ae5e2-c01a-44d3-b520-dd7d5a06ed4a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:28.173354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1167/2318] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:03:28.177398 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f3382-b310-4876-878d-122cea646086 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:28.187187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f3382-b310-4876-878d-122cea646086 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:28.192350 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316aa6f2-0f86-49fa-b2b2-cb8b4807f268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:28.193009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316aa6f2-0f86-49fa-b2b2-cb8b4807f268 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:28.193291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316aa6f2-0f86-49fa-b2b2-cb8b4807f268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:28.193616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316aa6f2-0f86-49fa-b2b2-cb8b4807f268 admin admin] PATH_INFO: `/v3/projects/9c1e529b4a644299a617363fe78049eb/users/78ba6235b4fd4730a64345f87853c781/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:28.214615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316aa6f2-0f86-49fa-b2b2-cb8b4807f268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1e529b4a644299a617363fe78049eb, user_id=78ba6235b4fd4730a64345f87853c781, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:28.217298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316aa6f2-0f86-49fa-b2b2-cb8b4807f268 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:28.230510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1152/2319] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:03:28 2022] PUT /identity/v3/projects/9c1e529b4a644299a617363fe78049eb/users/78ba6235b4fd4730a64345f87853c781/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:03:28.240760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a4ad-b30d-47f6-b800-fc8017524db3 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:28.241158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a4ad-b30d-47f6-b800-fc8017524db3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:28.241515 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a4ad-b30d-47f6-b800-fc8017524db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:28.270835 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2905a4ad-b30d-47f6-b800-fc8017524db3 None None] MFA Rules not processed for user `78ba6235b4fd4730a64345f87853c781`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:28.272077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2905a4ad-b30d-47f6-b800-fc8017524db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:28.345124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1168/2320] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:03:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:03:28.356688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316aa6f2-0f86-49fa-b2b2-cb8b4807f268 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:28.358401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-316aa6f2-0f86-49fa-b2b2-cb8b4807f268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:28.382289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:03:28.424811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316aa6f2-0f86-49fa-b2b2-cb8b4807f268 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:28.429603 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800919d-d14a-46e3-bf9d-c96374efd3b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:28.430242 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800919d-d14a-46e3-bf9d-c96374efd3b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:28.430531 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800919d-d14a-46e3-bf9d-c96374efd3b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:28.430834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800919d-d14a-46e3-bf9d-c96374efd3b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:28.442025 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800919d-d14a-46e3-bf9d-c96374efd3b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:28.443532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800919d-d14a-46e3-bf9d-c96374efd3b4 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:28.453374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1153/2321] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:03:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:03:29.825206 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7608a-1292-4e80-b742-c2479ee2ef79 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:29.825754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7608a-1292-4e80-b742-c2479ee2ef79 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:29.825754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7608a-1292-4e80-b742-c2479ee2ef79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:29.845135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f6d7608a-1292-4e80-b742-c2479ee2ef79 None None] MFA Rules not processed for user `78ba6235b4fd4730a64345f87853c781`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:29.847222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f6d7608a-1292-4e80-b742-c2479ee2ef79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:29.854023 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1169/2322] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:03:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:03:34.890818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79154-2f92-4549-93f3-05ad3b9e0a3e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:34.891970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79154-2f92-4549-93f3-05ad3b9e0a3e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:34.892405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79154-2f92-4549-93f3-05ad3b9e0a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:34.920191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-add79154-2f92-4549-93f3-05ad3b9e0a3e None None] MFA Rules not processed for user `352d60372ba249e38ea4aa2d62723f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:34.926297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-add79154-2f92-4549-93f3-05ad3b9e0a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:34.992451 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1154/2323] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:03:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:03:39.839545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:03:44.984460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=42 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:03:52.485201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7608a-1292-4e80-b742-c2479ee2ef79 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:52.489128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f6d7608a-1292-4e80-b742-c2479ee2ef79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:52.592317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7608a-1292-4e80-b742-c2479ee2ef79 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:52.597131 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:52.597986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:52.598277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:52.599637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] PATH_INFO: `/v3/users/518ad5eb820e44aa9689215e6038a521` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:52.600921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=518ad5eb820e44aa9689215e6038a521)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:52.602908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:52.682030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '518ad5eb820e44aa9689215e6038a521'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:52.682566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '518ad5eb820e44aa9689215e6038a521'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:52.699313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '518ad5eb820e44aa9689215e6038a521'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:52.709672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '518ad5eb820e44aa9689215e6038a521'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:52.725298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '518ad5eb820e44aa9689215e6038a521'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:52.743036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1170/2324] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:03:52 2022] DELETE /identity/v3/users/518ad5eb820e44aa9689215e6038a521 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:03:52.994487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1ff9e-44e6-4215-b939-e56e322b0277 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:52.995052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1ff9e-44e6-4215-b939-e56e322b0277 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:52.995156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1ff9e-44e6-4215-b939-e56e322b0277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:53.017601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-56c1ff9e-44e6-4215-b939-e56e322b0277 None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:53.020104 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-56c1ff9e-44e6-4215-b939-e56e322b0277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:53.125146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1155/2325] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 04:03:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:03:53.161656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:53.164550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:53.297520 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ffb75-5b17-449f-89b7-0580d31b07fe admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:53.306483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9ef2a-e154-4ffd-bef3-7e7bd8333c94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:53.307301 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9ef2a-e154-4ffd-bef3-7e7bd8333c94 service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:53.307691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9ef2a-e154-4ffd-bef3-7e7bd8333c94 service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:53.308070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9ef2a-e154-4ffd-bef3-7e7bd8333c94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:53.325714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9ef2a-e154-4ffd-bef3-7e7bd8333c94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:53.327632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9ef2a-e154-4ffd-bef3-7e7bd8333c94 service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:53.337260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1171/2326] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:03:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:03:53.443515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1ff9e-44e6-4215-b939-e56e322b0277 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:53.445537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-56c1ff9e-44e6-4215-b939-e56e322b0277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:53.552788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1ff9e-44e6-4215-b939-e56e322b0277 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:53.573212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:53.574335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:53.574972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:53.575349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] PATH_INFO: `/v3/projects/3e4289e377d24036b968d0ae71c61dcc` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:53.578109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e4289e377d24036b968d0ae71c61dcc)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:53.580926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:53.674407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e4289e377d24036b968d0ae71c61dcc', 'request_id': 'req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:53.680603 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e4289e377d24036b968d0ae71c61dcc', 'request_id': 'req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:53.691681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e4289e377d24036b968d0ae71c61dcc', 'request_id': 'req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:53.711302 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] The token cache is being invalidate because project 3e4289e377d24036b968d0ae71c61dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:03:53.711905 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:53.712795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:53.714531 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1156/2327] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:03:53 2022] DELETE /identity/v3/projects/3e4289e377d24036b968d0ae71c61dcc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:03:54.069673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee1b8-f407-4902-abca-362853e2a4de None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:54.070129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee1b8-f407-4902-abca-362853e2a4de None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:54.070221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee1b8-f407-4902-abca-362853e2a4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:54.090264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0b9ee1b8-f407-4902-abca-362853e2a4de None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:54.091526 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0b9ee1b8-f407-4902-abca-362853e2a4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:54.096942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:03:54.156574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1172/2328] 10.222.0.46 () {58 vars in 967 bytes} [Thu Jul 28 04:03:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:03:54.169928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:54.171388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:54.212002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:03:54.242758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b9911-6eba-4dbf-8c22-fda3deb2a16b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:54.247707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fef36-2493-4335-a383-fb32390850f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:54.248318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fef36-2493-4335-a383-fb32390850f1 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:54.248598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fef36-2493-4335-a383-fb32390850f1 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:54.248872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fef36-2493-4335-a383-fb32390850f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:54.257336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fef36-2493-4335-a383-fb32390850f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:54.258960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fef36-2493-4335-a383-fb32390850f1 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:54.266960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1157/2329] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:03:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:03:54.485238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c45841-4ed6-4366-8301-564dcb49c467 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:54.485531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c45841-4ed6-4366-8301-564dcb49c467 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:54.485800 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c45841-4ed6-4366-8301-564dcb49c467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:54.505730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-13c45841-4ed6-4366-8301-564dcb49c467 None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:54.507026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-13c45841-4ed6-4366-8301-564dcb49c467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:54.514098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1173/2330] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 04:03:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:03:54.525810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fef36-2493-4335-a383-fb32390850f1 service swift] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:54.536954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fef36-2493-4335-a383-fb32390850f1 service swift] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:54.542330 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db01cb8-1443-4864-8c12-6a68a4887d64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:54.543004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db01cb8-1443-4864-8c12-6a68a4887d64 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:54.543290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db01cb8-1443-4864-8c12-6a68a4887d64 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:54.543586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db01cb8-1443-4864-8c12-6a68a4887d64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:54.552720 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db01cb8-1443-4864-8c12-6a68a4887d64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:54.554310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db01cb8-1443-4864-8c12-6a68a4887d64 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:54.562563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1158/2331] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:03:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:03:54.882164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294900e-6bf7-4785-8d68-bea6b1dbc7b4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:54.882453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294900e-6bf7-4785-8d68-bea6b1dbc7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:54.882731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294900e-6bf7-4785-8d68-bea6b1dbc7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:54.901289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1294900e-6bf7-4785-8d68-bea6b1dbc7b4 None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:54.902469 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1294900e-6bf7-4785-8d68-bea6b1dbc7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:54.909543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1174/2332] 10.222.0.46 () {58 vars in 967 bytes} [Thu Jul 28 04:03:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:03:54.922741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db01cb8-1443-4864-8c12-6a68a4887d64 service swift] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:54.934544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db01cb8-1443-4864-8c12-6a68a4887d64 service swift] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:54.940920 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322dc69-0a24-4edd-9edf-653fa2110ba9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:54.941607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322dc69-0a24-4edd-9edf-653fa2110ba9 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:54.941892 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322dc69-0a24-4edd-9edf-653fa2110ba9 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:54.942169 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322dc69-0a24-4edd-9edf-653fa2110ba9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:54.951064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322dc69-0a24-4edd-9edf-653fa2110ba9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:54.952628 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322dc69-0a24-4edd-9edf-653fa2110ba9 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:54.961277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1159/2333] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:03:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:03:57.059625 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294900e-6bf7-4785-8d68-bea6b1dbc7b4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:57.062493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1294900e-6bf7-4785-8d68-bea6b1dbc7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:57.084218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322dc69-0a24-4edd-9edf-653fa2110ba9 service swift] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:57.085673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9322dc69-0a24-4edd-9edf-653fa2110ba9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:57.119104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294900e-6bf7-4785-8d68-bea6b1dbc7b4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:57.122676 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf32cb-195b-447f-aab3-947d95bcf3fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:57.123312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf32cb-195b-447f-aab3-947d95bcf3fa service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:57.123593 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf32cb-195b-447f-aab3-947d95bcf3fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:57.123867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf32cb-195b-447f-aab3-947d95bcf3fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:57.125601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-55bf32cb-195b-447f-aab3-947d95bcf3fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:57.150878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322dc69-0a24-4edd-9edf-653fa2110ba9 service swift] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:57.155703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:57.156421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:57.156788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:57.157102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] PATH_INFO: `/v3/users/78ba6235b4fd4730a64345f87853c781` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:57.158372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ba6235b4fd4730a64345f87853c781)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:57.160317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:57.191461 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf32cb-195b-447f-aab3-947d95bcf3fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:57.193322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf32cb-195b-447f-aab3-947d95bcf3fa service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:57.204244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1175/2334] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:03:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:03:57.210765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ba6235b4fd4730a64345f87853c781'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:57.211448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ba6235b4fd4730a64345f87853c781'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:57.218902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ba6235b4fd4730a64345f87853c781'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:57.230890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ba6235b4fd4730a64345f87853c781'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:57.237734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ba6235b4fd4730a64345f87853c781'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:57.243987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1160/2335] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:03:57 2022] DELETE /identity/v3/users/78ba6235b4fd4730a64345f87853c781 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:03:57.248657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075d100-e1ce-4407-a5b9-6e592a9f442d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:57.248945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075d100-e1ce-4407-a5b9-6e592a9f442d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:57.249220 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075d100-e1ce-4407-a5b9-6e592a9f442d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:57.268871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0075d100-e1ce-4407-a5b9-6e592a9f442d None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:03:57.270093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0075d100-e1ce-4407-a5b9-6e592a9f442d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:57.319016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:03:57.334323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1176/2336] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:03:57.344118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:57.346582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:57.368656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:03:57.411494 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaed173-af1c-447d-b487-4bb406337852 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:57.415479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c42e3-2b30-4e27-aeb3-aa709011d89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:57.416126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c42e3-2b30-4e27-aeb3-aa709011d89e service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:57.416412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c42e3-2b30-4e27-aeb3-aa709011d89e service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:57.416689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c42e3-2b30-4e27-aeb3-aa709011d89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:57.426041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c42e3-2b30-4e27-aeb3-aa709011d89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:57.427524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c42e3-2b30-4e27-aeb3-aa709011d89e service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:57.437476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1161/2337] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:03:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:03:57.648646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075d100-e1ce-4407-a5b9-6e592a9f442d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:03:57.650921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0075d100-e1ce-4407-a5b9-6e592a9f442d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:03:57.666964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075d100-e1ce-4407-a5b9-6e592a9f442d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:03:57.675102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:03:57.676072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:03:57.676424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:03:57.676794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] PATH_INFO: `/v3/projects/9c1e529b4a644299a617363fe78049eb` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:03:57.678066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1e529b4a644299a617363fe78049eb)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:03:57.680399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:03:57.753100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1e529b4a644299a617363fe78049eb', 'request_id': 'req-939e06a9-150c-4c8c-b1a6-9df091d22cc2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:57.772216 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1e529b4a644299a617363fe78049eb', 'request_id': 'req-939e06a9-150c-4c8c-b1a6-9df091d22cc2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:57.778849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1e529b4a644299a617363fe78049eb', 'request_id': 'req-939e06a9-150c-4c8c-b1a6-9df091d22cc2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:57.792460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] The token cache is being invalidate because project 9c1e529b4a644299a617363fe78049eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:03:57.792686 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:57.793321 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-939e06a9-150c-4c8c-b1a6-9df091d22cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:03:57.794876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1177/2338] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:03:57 2022] DELETE /identity/v3/projects/9c1e529b4a644299a617363fe78049eb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:04:01.961268 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1112-1e2e-494e-b96a-fdbee5d59145 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:04:01.961832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1112-1e2e-494e-b96a-fdbee5d59145 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:04:01.961832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1112-1e2e-494e-b96a-fdbee5d59145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:04:01.980493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5f6f1112-1e2e-494e-b96a-fdbee5d59145 None None] MFA Rules not processed for user `c3308761e9d2495e95e127bbc7b9c5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:04:01.981686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5f6f1112-1e2e-494e-b96a-fdbee5d59145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:02.035224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1162/2339] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:04:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:04:07.789293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:04:12.028562 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=43 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:04:26.547451 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bcc03-6646-4e0f-83e2-2862455af30b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:04:26.547950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bcc03-6646-4e0f-83e2-2862455af30b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:04:26.548056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bcc03-6646-4e0f-83e2-2862455af30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:04:26.567313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-888bcc03-6646-4e0f-83e2-2862455af30b None None] MFA Rules not processed for user `72dbbd19e4454bee84fe5bc797497bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:04:26.568427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-888bcc03-6646-4e0f-83e2-2862455af30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:26.625366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1178/2340] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:04:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:04:29.303193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f1112-1e2e-494e-b96a-fdbee5d59145 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:04:29.304616 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5f6f1112-1e2e-494e-b96a-fdbee5d59145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:29.362585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f1112-1e2e-494e-b96a-fdbee5d59145 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:04:29.366073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c4251-9ce5-4d11-916d-30bafc8d7e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:04:29.366677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c4251-9ce5-4d11-916d-30bafc8d7e02 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:04:29.366995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c4251-9ce5-4d11-916d-30bafc8d7e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:04:29.367267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c4251-9ce5-4d11-916d-30bafc8d7e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:04:29.368987 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-092c4251-9ce5-4d11-916d-30bafc8d7e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:29.437573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c4251-9ce5-4d11-916d-30bafc8d7e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:04:29.439138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c4251-9ce5-4d11-916d-30bafc8d7e02 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:04:29.450011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1163/2341] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:04:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:04:36.618460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=25 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:04:39.445247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=54 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:04:48.438182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bcc03-6646-4e0f-83e2-2862455af30b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:04:48.440209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-888bcc03-6646-4e0f-83e2-2862455af30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:48.495617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bcc03-6646-4e0f-83e2-2862455af30b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:04:48.499183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3031099-d194-495d-b021-8adefa54857b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:04:48.499788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3031099-d194-495d-b021-8adefa54857b service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:04:48.500068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3031099-d194-495d-b021-8adefa54857b service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:04:48.500343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3031099-d194-495d-b021-8adefa54857b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:04:48.502853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e3031099-d194-495d-b021-8adefa54857b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:48.561262 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3031099-d194-495d-b021-8adefa54857b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:04:48.562710 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3031099-d194-495d-b021-8adefa54857b service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:04:48.572843 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1179/2342] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:04:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:04:48.899797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c4251-9ce5-4d11-916d-30bafc8d7e02 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:04:48.901386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-092c4251-9ce5-4d11-916d-30bafc8d7e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:48.912149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c4251-9ce5-4d11-916d-30bafc8d7e02 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:04:48.915719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f058a9-344c-4298-86f4-dc2eb73473bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:04:48.916321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f058a9-344c-4298-86f4-dc2eb73473bc service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:04:48.916602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f058a9-344c-4298-86f4-dc2eb73473bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:04:48.916875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f058a9-344c-4298-86f4-dc2eb73473bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:04:48.918532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-58f058a9-344c-4298-86f4-dc2eb73473bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:48.932747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f058a9-344c-4298-86f4-dc2eb73473bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:04:48.934161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f058a9-344c-4298-86f4-dc2eb73473bc service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:04:48.945547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1164/2343] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:04:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:04:55.463921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53395b-4afe-40dd-a5a3-07a2466227b2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:04:55.464457 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53395b-4afe-40dd-a5a3-07a2466227b2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:04:55.464563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53395b-4afe-40dd-a5a3-07a2466227b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:04:55.483836 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8d53395b-4afe-40dd-a5a3-07a2466227b2 None None] MFA Rules not processed for user `7112379fb6794027a7c1519b2f77a5d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:04:55.485022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8d53395b-4afe-40dd-a5a3-07a2466227b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:55.492358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1180/2344] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:04:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:04:55.796138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f058a9-344c-4298-86f4-dc2eb73473bc service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:04:55.804875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f058a9-344c-4298-86f4-dc2eb73473bc service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:04:55.808499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71e189-0054-4498-8e06-05a453853f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:04:55.809102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71e189-0054-4498-8e06-05a453853f98 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:04:55.809383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71e189-0054-4498-8e06-05a453853f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:04:55.809653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71e189-0054-4498-8e06-05a453853f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:04:55.811429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ce71e189-0054-4498-8e06-05a453853f98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:55.821340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71e189-0054-4498-8e06-05a453853f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:04:55.822743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71e189-0054-4498-8e06-05a453853f98 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:04:55.830473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1165/2345] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:04:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:04:56.135974 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53395b-4afe-40dd-a5a3-07a2466227b2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:04:56.137410 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8d53395b-4afe-40dd-a5a3-07a2466227b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:56.211550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53395b-4afe-40dd-a5a3-07a2466227b2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:04:56.216521 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caa08f-00c5-4a5a-91cb-e00d2cb5e7d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:04:56.217135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa08f-00c5-4a5a-91cb-e00d2cb5e7d7 service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:04:56.217419 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa08f-00c5-4a5a-91cb-e00d2cb5e7d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:04:56.217695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa08f-00c5-4a5a-91cb-e00d2cb5e7d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:04:56.219450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-48caa08f-00c5-4a5a-91cb-e00d2cb5e7d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:56.234959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48caa08f-00c5-4a5a-91cb-e00d2cb5e7d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:04:56.236559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48caa08f-00c5-4a5a-91cb-e00d2cb5e7d7 service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:04:56.247694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1181/2346] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:04:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:04:57.943400 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71e189-0054-4498-8e06-05a453853f98 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:04:57.952365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71e189-0054-4498-8e06-05a453853f98 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:04:57.956181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd47bd8-25d4-4ae7-9191-1731a85f69d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:04:57.956813 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47bd8-25d4-4ae7-9191-1731a85f69d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:04:57.957099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47bd8-25d4-4ae7-9191-1731a85f69d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:04:57.957379 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47bd8-25d4-4ae7-9191-1731a85f69d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:04:57.959204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7bd47bd8-25d4-4ae7-9191-1731a85f69d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:57.969557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd47bd8-25d4-4ae7-9191-1731a85f69d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:04:57.971039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd47bd8-25d4-4ae7-9191-1731a85f69d8 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:04:57.979775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1166/2347] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:04:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:04:59.099524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caa08f-00c5-4a5a-91cb-e00d2cb5e7d7 service placement] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:04:59.101363 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-48caa08f-00c5-4a5a-91cb-e00d2cb5e7d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:59.116548 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caa08f-00c5-4a5a-91cb-e00d2cb5e7d7 service placement] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:04:59.122116 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70653f68-d5e4-4dc7-901d-7c8e92a95c73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:04:59.122735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70653f68-d5e4-4dc7-901d-7c8e92a95c73 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:04:59.123050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70653f68-d5e4-4dc7-901d-7c8e92a95c73 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:04:59.123330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70653f68-d5e4-4dc7-901d-7c8e92a95c73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:04:59.125090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-70653f68-d5e4-4dc7-901d-7c8e92a95c73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:04:59.136727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70653f68-d5e4-4dc7-901d-7c8e92a95c73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:04:59.138311 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70653f68-d5e4-4dc7-901d-7c8e92a95c73 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:04:59.148212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1182/2348] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:04:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:05:06.885414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd47bd8-25d4-4ae7-9191-1731a85f69d8 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:06.898499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd47bd8-25d4-4ae7-9191-1731a85f69d8 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:06.904220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515d1dc-34e7-4550-bce9-530ba456183d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:06.904874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515d1dc-34e7-4550-bce9-530ba456183d service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:06.905161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515d1dc-34e7-4550-bce9-530ba456183d service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:06.907095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515d1dc-34e7-4550-bce9-530ba456183d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:06.909434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8515d1dc-34e7-4550-bce9-530ba456183d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:06.922524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515d1dc-34e7-4550-bce9-530ba456183d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:06.924671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515d1dc-34e7-4550-bce9-530ba456183d service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:06.935795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1167/2349] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:05:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:05:09.143757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:05:09.791692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70653f68-d5e4-4dc7-901d-7c8e92a95c73 service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:09.801175 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70653f68-d5e4-4dc7-901d-7c8e92a95c73 service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:09.804739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01984881-f3ba-486a-8a1f-d014e632a46f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:09.805343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01984881-f3ba-486a-8a1f-d014e632a46f service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:09.805621 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01984881-f3ba-486a-8a1f-d014e632a46f service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:09.805894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01984881-f3ba-486a-8a1f-d014e632a46f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:09.807700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-01984881-f3ba-486a-8a1f-d014e632a46f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:09.819518 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01984881-f3ba-486a-8a1f-d014e632a46f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:09.821009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01984881-f3ba-486a-8a1f-d014e632a46f service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:09.830853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1183/2350] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:05:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:05:16.931153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=24 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:05:19.071322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189839f3-d232-4489-a62e-fbe94cdb8d9b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:19.071943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189839f3-d232-4489-a62e-fbe94cdb8d9b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:19.071943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189839f3-d232-4489-a62e-fbe94cdb8d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:19.091113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-189839f3-d232-4489-a62e-fbe94cdb8d9b None None] MFA Rules not processed for user `c3308761e9d2495e95e127bbc7b9c5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:05:19.092314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-189839f3-d232-4489-a62e-fbe94cdb8d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:19.100330 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1168/2351] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:05:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:05:19.410517 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01984881-f3ba-486a-8a1f-d014e632a46f service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:19.420013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01984881-f3ba-486a-8a1f-d014e632a46f service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:19.424271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16696cef-e65a-4996-9b39-89ea4757605b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:19.424874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16696cef-e65a-4996-9b39-89ea4757605b service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:19.425161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16696cef-e65a-4996-9b39-89ea4757605b service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:19.425440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16696cef-e65a-4996-9b39-89ea4757605b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:19.427227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-16696cef-e65a-4996-9b39-89ea4757605b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:19.436538 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16696cef-e65a-4996-9b39-89ea4757605b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:19.437973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16696cef-e65a-4996-9b39-89ea4757605b service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:19.446906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1184/2352] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:05:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:05:29.086357 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:05:29.159533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189839f3-d232-4489-a62e-fbe94cdb8d9b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:29.169715 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189839f3-d232-4489-a62e-fbe94cdb8d9b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:29.173906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8253b30-2053-4f45-bb09-27830acf10a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:29.174812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8253b30-2053-4f45-bb09-27830acf10a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:29.175110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8253b30-2053-4f45-bb09-27830acf10a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:29.175382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8253b30-2053-4f45-bb09-27830acf10a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:29.177182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a8253b30-2053-4f45-bb09-27830acf10a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:29.241469 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8253b30-2053-4f45-bb09-27830acf10a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:29.243028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8253b30-2053-4f45-bb09-27830acf10a4 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:29.253068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1169/2353] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:05:29.443244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=12 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:05:31.769866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16696cef-e65a-4996-9b39-89ea4757605b service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:31.777222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-16696cef-e65a-4996-9b39-89ea4757605b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:31.802650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16696cef-e65a-4996-9b39-89ea4757605b service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:31.808713 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e0029-ca5e-4b88-b984-81a20e1d0969 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:31.809438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e0029-ca5e-4b88-b984-81a20e1d0969 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:31.809791 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e0029-ca5e-4b88-b984-81a20e1d0969 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:31.810136 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e0029-ca5e-4b88-b984-81a20e1d0969 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:31.813646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0f9e0029-ca5e-4b88-b984-81a20e1d0969 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:31.839289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e0029-ca5e-4b88-b984-81a20e1d0969 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:31.840889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e0029-ca5e-4b88-b984-81a20e1d0969 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:31.856775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1185/2354] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:05:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:05:32.577817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8253b30-2053-4f45-bb09-27830acf10a4 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:32.596032 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8253b30-2053-4f45-bb09-27830acf10a4 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:32.613590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843e1d9-f04a-4e88-be11-cd46d22a2f01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:32.614562 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843e1d9-f04a-4e88-be11-cd46d22a2f01 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:32.615071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843e1d9-f04a-4e88-be11-cd46d22a2f01 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:32.615661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843e1d9-f04a-4e88-be11-cd46d22a2f01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:32.617896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e843e1d9-f04a-4e88-be11-cd46d22a2f01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:32.630709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843e1d9-f04a-4e88-be11-cd46d22a2f01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:32.632573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843e1d9-f04a-4e88-be11-cd46d22a2f01 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:32.644508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1170/2355] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:05:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:05:32.645779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e0029-ca5e-4b88-b984-81a20e1d0969 service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:32.649106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0f9e0029-ca5e-4b88-b984-81a20e1d0969 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:32.665391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e0029-ca5e-4b88-b984-81a20e1d0969 service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:32.670163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff2eb1-0b21-4954-85e4-57a08d30e75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:32.670831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff2eb1-0b21-4954-85e4-57a08d30e75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:32.671127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff2eb1-0b21-4954-85e4-57a08d30e75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:32.671407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff2eb1-0b21-4954-85e4-57a08d30e75a admin admin] PATH_INFO: `/v3/users/72dbbd19e4454bee84fe5bc797497bd0` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:32.672602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff2eb1-0b21-4954-85e4-57a08d30e75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=72dbbd19e4454bee84fe5bc797497bd0)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:32.675174 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff2eb1-0b21-4954-85e4-57a08d30e75a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:32.721867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-40ff2eb1-0b21-4954-85e4-57a08d30e75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72dbbd19e4454bee84fe5bc797497bd0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:05:32.722526 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-40ff2eb1-0b21-4954-85e4-57a08d30e75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72dbbd19e4454bee84fe5bc797497bd0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:05:32.734198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-40ff2eb1-0b21-4954-85e4-57a08d30e75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72dbbd19e4454bee84fe5bc797497bd0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:05:32.741791 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-40ff2eb1-0b21-4954-85e4-57a08d30e75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72dbbd19e4454bee84fe5bc797497bd0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:05:32.749003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-40ff2eb1-0b21-4954-85e4-57a08d30e75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72dbbd19e4454bee84fe5bc797497bd0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:05:32.754912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1186/2356] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:05:32 2022] DELETE /identity/v3/users/72dbbd19e4454bee84fe5bc797497bd0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:05:33.017642 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843e1d9-f04a-4e88-be11-cd46d22a2f01 service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:33.028607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843e1d9-f04a-4e88-be11-cd46d22a2f01 service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:33.087266 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:33.087910 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:33.088194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:33.088471 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] PATH_INFO: `/v3/projects/6ca5212b1a1743f2910407d7b8ba7aaf` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:33.089635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca5212b1a1743f2910407d7b8ba7aaf)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:33.091586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:33.140153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca5212b1a1743f2910407d7b8ba7aaf', 'request_id': 'req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:05:33.146180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca5212b1a1743f2910407d7b8ba7aaf', 'request_id': 'req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:05:33.153748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca5212b1a1743f2910407d7b8ba7aaf', 'request_id': 'req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:05:33.166822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] The token cache is being invalidate because project 6ca5212b1a1743f2910407d7b8ba7aaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:05:33.167231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:05:33.168100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f6c8620d-03f2-490a-8a19-cc67b55bfc8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:05:33.169856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1171/2357] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:05:33 2022] DELETE /identity/v3/projects/6ca5212b1a1743f2910407d7b8ba7aaf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:05:33.176113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46774b8-6a34-4b81-a4ea-a208d6d5b75e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:33.176603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46774b8-6a34-4b81-a4ea-a208d6d5b75e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:33.176806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46774b8-6a34-4b81-a4ea-a208d6d5b75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:33.199013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f46774b8-6a34-4b81-a4ea-a208d6d5b75e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:05:33.200220 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f46774b8-6a34-4b81-a4ea-a208d6d5b75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:33.271766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1187/2358] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:05:33.279499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6934984-f0bf-439a-bfb4-fe2a61461e74 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:33.279793 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6934984-f0bf-439a-bfb4-fe2a61461e74 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:33.280069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6934984-f0bf-439a-bfb4-fe2a61461e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:33.301110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f6934984-f0bf-439a-bfb4-fe2a61461e74 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:05:33.302307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f6934984-f0bf-439a-bfb4-fe2a61461e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:33.310097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1172/2359] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:05:33.315361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46774b8-6a34-4b81-a4ea-a208d6d5b75e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:33.325053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46774b8-6a34-4b81-a4ea-a208d6d5b75e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:33.330836 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819874b5-bf9b-4dba-85f2-68a97f3f36fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:33.331450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819874b5-bf9b-4dba-85f2-68a97f3f36fe admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:33.331764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819874b5-bf9b-4dba-85f2-68a97f3f36fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:33.332040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819874b5-bf9b-4dba-85f2-68a97f3f36fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:33.332739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819874b5-bf9b-4dba-85f2-68a97f3f36fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:05:33.333045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819874b5-bf9b-4dba-85f2-68a97f3f36fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:33.334638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819874b5-bf9b-4dba-85f2-68a97f3f36fe admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:33.347067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1188/2360] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:05:33.351601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6934984-f0bf-439a-bfb4-fe2a61461e74 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:33.357273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:05:33.362664 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6934984-f0bf-439a-bfb4-fe2a61461e74 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:33.368531 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57d339-0d64-4a6e-90cd-94f72d20243f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:33.369247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57d339-0d64-4a6e-90cd-94f72d20243f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:33.369739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57d339-0d64-4a6e-90cd-94f72d20243f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:33.370194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57d339-0d64-4a6e-90cd-94f72d20243f admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:33.370981 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57d339-0d64-4a6e-90cd-94f72d20243f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:33.373114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57d339-0d64-4a6e-90cd-94f72d20243f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:33.401693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1173/2361] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:05:33 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:05:33.405807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819874b5-bf9b-4dba-85f2-68a97f3f36fe admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:33.416607 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819874b5-bf9b-4dba-85f2-68a97f3f36fe admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:33.459826 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:05:33.474481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c4e26-f0b9-4ba0-8a13-dc05ac0bce58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:33.475238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c4e26-f0b9-4ba0-8a13-dc05ac0bce58 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:33.475525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c4e26-f0b9-4ba0-8a13-dc05ac0bce58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:33.475802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c4e26-f0b9-4ba0-8a13-dc05ac0bce58 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:33.476471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c4e26-f0b9-4ba0-8a13-dc05ac0bce58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:33.478380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c4e26-f0b9-4ba0-8a13-dc05ac0bce58 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:33.500180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1189/2362] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:05:33 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:05:33.504450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57d339-0d64-4a6e-90cd-94f72d20243f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:33.514610 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57d339-0d64-4a6e-90cd-94f72d20243f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:33.519351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea98e5f-ef76-4db1-8d69-f528473e7ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:33.519956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea98e5f-ef76-4db1-8d69-f528473e7ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:33.520238 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea98e5f-ef76-4db1-8d69-f528473e7ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:33.520516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea98e5f-ef76-4db1-8d69-f528473e7ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:33.521213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea98e5f-ef76-4db1-8d69-f528473e7ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:33.522820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea98e5f-ef76-4db1-8d69-f528473e7ff5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:33.533459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1174/2363] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:05:33.537806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c4e26-f0b9-4ba0-8a13-dc05ac0bce58 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:33.547820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c4e26-f0b9-4ba0-8a13-dc05ac0bce58 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:33.553150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e0650-07cb-4c12-834a-832dd899129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:33.553763 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e0650-07cb-4c12-834a-832dd899129f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:33.554045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e0650-07cb-4c12-834a-832dd899129f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:33.554319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e0650-07cb-4c12-834a-832dd899129f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:33.555053 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e0650-07cb-4c12-834a-832dd899129f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:33.556668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e0650-07cb-4c12-834a-832dd899129f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:33.567400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1190/2364] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:05:33.571665 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea98e5f-ef76-4db1-8d69-f528473e7ff5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:33.581695 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea98e5f-ef76-4db1-8d69-f528473e7ff5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:33.586423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691abbea-7f08-4376-83be-b09796ad88db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:33.587288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691abbea-7f08-4376-83be-b09796ad88db admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:33.587568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691abbea-7f08-4376-83be-b09796ad88db admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:33.587844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691abbea-7f08-4376-83be-b09796ad88db admin admin] PATH_INFO: `/v3/projects/901cf93dff224196b6727a29f0038bdb/users/b7e6ceb0f76d41ce962efd065ffa3d5a/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:33.605320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691abbea-7f08-4376-83be-b09796ad88db admin admin] RBAC: Authorizing `identity:create_grant(project_id=901cf93dff224196b6727a29f0038bdb, user_id=b7e6ceb0f76d41ce962efd065ffa3d5a, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:33.608196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691abbea-7f08-4376-83be-b09796ad88db admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:33.625841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1175/2365] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:05:33 2022] PUT /identity/v3/projects/901cf93dff224196b6727a29f0038bdb/users/b7e6ceb0f76d41ce962efd065ffa3d5a/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:05:33.631086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0df8b-90ec-46ec-88a7-9feee0b4a1ae None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:33.631465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0df8b-90ec-46ec-88a7-9feee0b4a1ae None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:33.631760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0df8b-90ec-46ec-88a7-9feee0b4a1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:33.650321 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-deb0df8b-90ec-46ec-88a7-9feee0b4a1ae None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:05:33.651559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-deb0df8b-90ec-46ec-88a7-9feee0b4a1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:33.718186 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1191/2366] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:05:33.728426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691abbea-7f08-4376-83be-b09796ad88db admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:33.730152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-691abbea-7f08-4376-83be-b09796ad88db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:33.787224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691abbea-7f08-4376-83be-b09796ad88db admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:33.790919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fc970-ca98-41a9-9769-062937d77899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:33.791535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fc970-ca98-41a9-9769-062937d77899 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:33.792759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fc970-ca98-41a9-9769-062937d77899 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:33.793036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fc970-ca98-41a9-9769-062937d77899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:33.802463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fc970-ca98-41a9-9769-062937d77899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:33.803924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fc970-ca98-41a9-9769-062937d77899 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:33.813258 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1176/2367] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:05:38.255939 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9c49-351b-476f-a17a-6ab4e08e2b2a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:38.256500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9c49-351b-476f-a17a-6ab4e08e2b2a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:38.256500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9c49-351b-476f-a17a-6ab4e08e2b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:38.297383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c97d9c49-351b-476f-a17a-6ab4e08e2b2a None None] MFA Rules not processed for user `b7e6ceb0f76d41ce962efd065ffa3d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:05:38.298689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c97d9c49-351b-476f-a17a-6ab4e08e2b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:38.376034 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1192/2368] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:05:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:05:38.395128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fc970-ca98-41a9-9769-062937d77899 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:05:38.396720 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-541fc970-ca98-41a9-9769-062937d77899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:38.484885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fc970-ca98-41a9-9769-062937d77899 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:05:38.490601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd19c3-9a8f-41fd-9913-0e61df124166 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:05:38.491276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd19c3-9a8f-41fd-9913-0e61df124166 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:38.491580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd19c3-9a8f-41fd-9913-0e61df124166 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:38.491842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd19c3-9a8f-41fd-9913-0e61df124166 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:38.500417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd19c3-9a8f-41fd-9913-0e61df124166 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:05:38.502132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd19c3-9a8f-41fd-9913-0e61df124166 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:05:38.512004 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1177/2369] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:05:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:05:39.817467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb954132-b4bd-4ff3-ac65-ce29226028de None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:05:39.817937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb954132-b4bd-4ff3-ac65-ce29226028de None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:05:39.818050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb954132-b4bd-4ff3-ac65-ce29226028de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:05:39.837225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bb954132-b4bd-4ff3-ac65-ce29226028de None None] MFA Rules not processed for user `b7e6ceb0f76d41ce962efd065ffa3d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:05:39.838436 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bb954132-b4bd-4ff3-ac65-ce29226028de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:05:39.847294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1193/2370] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:05:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:05:48.507463 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=82,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:05:49.831847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:11.180921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7d5a-ee9f-4c70-9efb-8f3208e753a3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:11.181522 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7d5a-ee9f-4c70-9efb-8f3208e753a3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:11.181522 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7d5a-ee9f-4c70-9efb-8f3208e753a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:11.200914 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ba4a7d5a-ee9f-4c70-9efb-8f3208e753a3 None None] MFA Rules not processed for user `352d60372ba249e38ea4aa2d62723f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:11.202018 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ba4a7d5a-ee9f-4c70-9efb-8f3208e753a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:11.258424 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1178/2371] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:15.037851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb954132-b4bd-4ff3-ac65-ce29226028de None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:15.039461 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bb954132-b4bd-4ff3-ac65-ce29226028de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:15.052488 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb954132-b4bd-4ff3-ac65-ce29226028de None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:15.056217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6fa04-f8d1-4ca5-9bc1-df18ee0c92ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:15.056912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6fa04-f8d1-4ca5-9bc1-df18ee0c92ab service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:15.057261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6fa04-f8d1-4ca5-9bc1-df18ee0c92ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:15.057599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6fa04-f8d1-4ca5-9bc1-df18ee0c92ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:15.059501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1be6fa04-f8d1-4ca5-9bc1-df18ee0c92ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:15.073307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6fa04-f8d1-4ca5-9bc1-df18ee0c92ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:15.074886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6fa04-f8d1-4ca5-9bc1-df18ee0c92ab service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:15.086004 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1194/2372] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:18.299707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a7d5a-ee9f-4c70-9efb-8f3208e753a3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:18.301243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ba4a7d5a-ee9f-4c70-9efb-8f3208e753a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:18.364875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6fa04-f8d1-4ca5-9bc1-df18ee0c92ab service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:18.366438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1be6fa04-f8d1-4ca5-9bc1-df18ee0c92ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:18.383573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6fa04-f8d1-4ca5-9bc1-df18ee0c92ab service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:18.383743 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a7d5a-ee9f-4c70-9efb-8f3208e753a3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:18.388311 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35298c57-19a4-4862-8c3d-28dce608c291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:18.388722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f270e1-e81a-4048-af41-9e9a6c41f60e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:18.389102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298c57-19a4-4862-8c3d-28dce608c291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:18.389346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f270e1-e81a-4048-af41-9e9a6c41f60e service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:18.389502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298c57-19a4-4862-8c3d-28dce608c291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:18.389632 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f270e1-e81a-4048-af41-9e9a6c41f60e service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:18.389895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298c57-19a4-4862-8c3d-28dce608c291 admin admin] PATH_INFO: `/v3/users/7112379fb6794027a7c1519b2f77a5d4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:18.390000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f270e1-e81a-4048-af41-9e9a6c41f60e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:18.391233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35298c57-19a4-4862-8c3d-28dce608c291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7112379fb6794027a7c1519b2f77a5d4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:18.391947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d1f270e1-e81a-4048-af41-9e9a6c41f60e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:18.393599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35298c57-19a4-4862-8c3d-28dce608c291 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:18.410239 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f270e1-e81a-4048-af41-9e9a6c41f60e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:18.412164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f270e1-e81a-4048-af41-9e9a6c41f60e service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:18.423622 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1179/2373] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:18.456900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-35298c57-19a4-4862-8c3d-28dce608c291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7112379fb6794027a7c1519b2f77a5d4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:18.457632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-35298c57-19a4-4862-8c3d-28dce608c291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7112379fb6794027a7c1519b2f77a5d4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:18.470592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-35298c57-19a4-4862-8c3d-28dce608c291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7112379fb6794027a7c1519b2f77a5d4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:18.483407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-35298c57-19a4-4862-8c3d-28dce608c291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7112379fb6794027a7c1519b2f77a5d4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:18.491522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-35298c57-19a4-4862-8c3d-28dce608c291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7112379fb6794027a7c1519b2f77a5d4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:18.497506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1195/2374] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:18 2022] DELETE /identity/v3/users/7112379fb6794027a7c1519b2f77a5d4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:18.677094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f270e1-e81a-4048-af41-9e9a6c41f60e service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:18.688832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f270e1-e81a-4048-af41-9e9a6c41f60e service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:18.751466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:18.752041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:18.752327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:18.752605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] PATH_INFO: `/v3/projects/5db0f08a14ac472fb3edf2107c9ccfc5` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:18.753796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db0f08a14ac472fb3edf2107c9ccfc5)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:18.755749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:18.793643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:18.803179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db0f08a14ac472fb3edf2107c9ccfc5', 'request_id': 'req-e5e12c63-c6f0-4111-90de-f98081435328'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:18.808730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db0f08a14ac472fb3edf2107c9ccfc5', 'request_id': 'req-e5e12c63-c6f0-4111-90de-f98081435328'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:18.815970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db0f08a14ac472fb3edf2107c9ccfc5', 'request_id': 'req-e5e12c63-c6f0-4111-90de-f98081435328'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:18.826847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] The token cache is being invalidate because project 5db0f08a14ac472fb3edf2107c9ccfc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:18.827137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:18.827772 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5e12c63-c6f0-4111-90de-f98081435328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:18.829901 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1180/2375] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:18 2022] DELETE /identity/v3/projects/5db0f08a14ac472fb3edf2107c9ccfc5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:18.835732 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54914-1127-4fef-a2b8-af6aebe31e81 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:18.836019 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54914-1127-4fef-a2b8-af6aebe31e81 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:18.836297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54914-1127-4fef-a2b8-af6aebe31e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:18.856253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c0c54914-1127-4fef-a2b8-af6aebe31e81 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:18.857454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c0c54914-1127-4fef-a2b8-af6aebe31e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:18.924807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1196/2376] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:18.933202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763ccc9-1ed2-4ffb-8f0f-8af8f43c4d6a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:18.933496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763ccc9-1ed2-4ffb-8f0f-8af8f43c4d6a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:18.933779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763ccc9-1ed2-4ffb-8f0f-8af8f43c4d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:18.954472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e763ccc9-1ed2-4ffb-8f0f-8af8f43c4d6a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:18.955730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e763ccc9-1ed2-4ffb-8f0f-8af8f43c4d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:18.963946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1181/2377] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:18.968176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c54914-1127-4fef-a2b8-af6aebe31e81 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:18.978506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c54914-1127-4fef-a2b8-af6aebe31e81 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:18.983159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06769ba-5e26-444d-ae21-6f37dce2ad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:18.983769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06769ba-5e26-444d-ae21-6f37dce2ad97 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:18.984051 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06769ba-5e26-444d-ae21-6f37dce2ad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:18.984323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06769ba-5e26-444d-ae21-6f37dce2ad97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:18.985277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06769ba-5e26-444d-ae21-6f37dce2ad97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:06:18.985584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06769ba-5e26-444d-ae21-6f37dce2ad97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:18.987198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06769ba-5e26-444d-ae21-6f37dce2ad97 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:18.998899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1197/2378] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:06:19.004035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763ccc9-1ed2-4ffb-8f0f-8af8f43c4d6a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.014179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763ccc9-1ed2-4ffb-8f0f-8af8f43c4d6a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:19.019588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585032a-2736-4401-8b82-fb8d0b298556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:19.020190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585032a-2736-4401-8b82-fb8d0b298556 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.020494 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585032a-2736-4401-8b82-fb8d0b298556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.020764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585032a-2736-4401-8b82-fb8d0b298556 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.021447 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585032a-2736-4401-8b82-fb8d0b298556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:19.023374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585032a-2736-4401-8b82-fb8d0b298556 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:19.049943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1182/2379] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:19 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:06:19.054248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06769ba-5e26-444d-ae21-6f37dce2ad97 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.064811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06769ba-5e26-444d-ae21-6f37dce2ad97 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:19.108921 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:19.123456 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634250f-8e88-4093-ac45-16f5887f71f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:19.124101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634250f-8e88-4093-ac45-16f5887f71f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.124393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634250f-8e88-4093-ac45-16f5887f71f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.124673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634250f-8e88-4093-ac45-16f5887f71f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.125934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634250f-8e88-4093-ac45-16f5887f71f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:19.128005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634250f-8e88-4093-ac45-16f5887f71f0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:19.155106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1198/2380] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:19 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:06:19.156932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585032a-2736-4401-8b82-fb8d0b298556 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.170073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585032a-2736-4401-8b82-fb8d0b298556 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:19.175798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca7d3a-277b-4aaf-8a62-b05cd376e4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:19.176559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca7d3a-277b-4aaf-8a62-b05cd376e4bd admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.176845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca7d3a-277b-4aaf-8a62-b05cd376e4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.177124 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca7d3a-277b-4aaf-8a62-b05cd376e4bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.177840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca7d3a-277b-4aaf-8a62-b05cd376e4bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:19.179493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca7d3a-277b-4aaf-8a62-b05cd376e4bd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:19.190679 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1183/2381] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:19.195002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634250f-8e88-4093-ac45-16f5887f71f0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.208418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634250f-8e88-4093-ac45-16f5887f71f0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:19.212976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b5941-67a4-4855-b549-acae109944eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:19.213613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b5941-67a4-4855-b549-acae109944eb admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.213898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b5941-67a4-4855-b549-acae109944eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.214174 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b5941-67a4-4855-b549-acae109944eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.214908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b5941-67a4-4855-b549-acae109944eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:19.216527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b5941-67a4-4855-b549-acae109944eb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:19.228896 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1199/2382] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:19.233108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca7d3a-277b-4aaf-8a62-b05cd376e4bd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.243947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca7d3a-277b-4aaf-8a62-b05cd376e4bd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:19.248853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6df111-688e-4d44-ae89-51421689941a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:19.249528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6df111-688e-4d44-ae89-51421689941a admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.249809 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6df111-688e-4d44-ae89-51421689941a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.250126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6df111-688e-4d44-ae89-51421689941a admin admin] PATH_INFO: `/v3/projects/e64ea134a7cb4983be82c17ddcc224d2/users/303caf433510436b917db3e63cdda0e2/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.268839 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6df111-688e-4d44-ae89-51421689941a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64ea134a7cb4983be82c17ddcc224d2, user_id=303caf433510436b917db3e63cdda0e2, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:19.271588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6df111-688e-4d44-ae89-51421689941a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:19.286521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1184/2383] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:19 2022] PUT /identity/v3/projects/e64ea134a7cb4983be82c17ddcc224d2/users/303caf433510436b917db3e63cdda0e2/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:19.298591 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80824-1301-4d14-abb3-b1df7ad94a14 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.298999 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80824-1301-4d14-abb3-b1df7ad94a14 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.299372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80824-1301-4d14-abb3-b1df7ad94a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.332287 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-82b80824-1301-4d14-abb3-b1df7ad94a14 None None] MFA Rules not processed for user `303caf433510436b917db3e63cdda0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:19.333536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-82b80824-1301-4d14-abb3-b1df7ad94a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:19.402009 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1200/2384] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:19.406261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6df111-688e-4d44-ae89-51421689941a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.468887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6df111-688e-4d44-ae89-51421689941a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:19.473524 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab0b3b-05b6-4a11-b00a-078e1163b010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:19.474131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab0b3b-05b6-4a11-b00a-078e1163b010 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.474411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab0b3b-05b6-4a11-b00a-078e1163b010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.474683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab0b3b-05b6-4a11-b00a-078e1163b010 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.475374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab0b3b-05b6-4a11-b00a-078e1163b010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:19.477260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab0b3b-05b6-4a11-b00a-078e1163b010 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:19.508014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1185/2385] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:19 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:19.509256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b80824-1301-4d14-abb3-b1df7ad94a14 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.520202 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b80824-1301-4d14-abb3-b1df7ad94a14 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:19.578830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87532298-c6ec-427b-956e-7fd499a83437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:19.579400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532298-c6ec-427b-956e-7fd499a83437 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.579883 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532298-c6ec-427b-956e-7fd499a83437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.580169 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532298-c6ec-427b-956e-7fd499a83437 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.580841 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87532298-c6ec-427b-956e-7fd499a83437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:19.582808 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87532298-c6ec-427b-956e-7fd499a83437 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:19.609049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1201/2386] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:19 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:19.610855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab0b3b-05b6-4a11-b00a-078e1163b010 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.621437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab0b3b-05b6-4a11-b00a-078e1163b010 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:19.626822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a02fe-7f84-4587-a04e-ddc19962890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:19.627440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a02fe-7f84-4587-a04e-ddc19962890f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.627731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a02fe-7f84-4587-a04e-ddc19962890f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.628037 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a02fe-7f84-4587-a04e-ddc19962890f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.628739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a02fe-7f84-4587-a04e-ddc19962890f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:19.630391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a02fe-7f84-4587-a04e-ddc19962890f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:19.642627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1186/2387] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:19.647156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87532298-c6ec-427b-956e-7fd499a83437 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.657284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87532298-c6ec-427b-956e-7fd499a83437 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:19.662156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f25da-d8cd-49d6-8763-1a21c80480ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:19.662839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f25da-d8cd-49d6-8763-1a21c80480ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.663127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f25da-d8cd-49d6-8763-1a21c80480ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.663404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f25da-d8cd-49d6-8763-1a21c80480ba admin admin] PATH_INFO: `/v3/projects/d50d0fe9270a44db8e3629f2fbe71111/users/df974507c2734fa5837fb26adcebcb52/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.678945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f25da-d8cd-49d6-8763-1a21c80480ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50d0fe9270a44db8e3629f2fbe71111, user_id=df974507c2734fa5837fb26adcebcb52, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:19.681575 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f25da-d8cd-49d6-8763-1a21c80480ba admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:19.694179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1202/2388] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:19 2022] PUT /identity/v3/projects/d50d0fe9270a44db8e3629f2fbe71111/users/df974507c2734fa5837fb26adcebcb52/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:19.698215 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a02fe-7f84-4587-a04e-ddc19962890f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.708722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a02fe-7f84-4587-a04e-ddc19962890f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:19.741224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:19.768014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848f601-e092-46ee-a409-c4bc2b40b37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:19.768552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f601-e092-46ee-a409-c4bc2b40b37a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.768837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f601-e092-46ee-a409-c4bc2b40b37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.769109 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f601-e092-46ee-a409-c4bc2b40b37a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.769790 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848f601-e092-46ee-a409-c4bc2b40b37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:19.771430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848f601-e092-46ee-a409-c4bc2b40b37a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:19.780881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1187/2389] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:06:19.785331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f25da-d8cd-49d6-8763-1a21c80480ba admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.795857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f25da-d8cd-49d6-8763-1a21c80480ba admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:19.800687 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8d703-91f1-4ffc-a284-e69936a630ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:19.801344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8d703-91f1-4ffc-a284-e69936a630ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.801631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8d703-91f1-4ffc-a284-e69936a630ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.801911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8d703-91f1-4ffc-a284-e69936a630ce admin admin] PATH_INFO: `/v3/projects/d50d0fe9270a44db8e3629f2fbe71111/users/df974507c2734fa5837fb26adcebcb52/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.803892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8d703-91f1-4ffc-a284-e69936a630ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50d0fe9270a44db8e3629f2fbe71111, user_id=df974507c2734fa5837fb26adcebcb52, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:19.806521 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8d703-91f1-4ffc-a284-e69936a630ce admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:19.817179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1203/2390] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:19 2022] PUT /identity/v3/projects/d50d0fe9270a44db8e3629f2fbe71111/users/df974507c2734fa5837fb26adcebcb52/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:19.827126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a653398-758f-4074-a9e6-706c45dee505 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:19.827442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a653398-758f-4074-a9e6-706c45dee505 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:19.827718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a653398-758f-4074-a9e6-706c45dee505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:19.855462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3a653398-758f-4074-a9e6-706c45dee505 None None] MFA Rules not processed for user `df974507c2734fa5837fb26adcebcb52`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:19.856728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3a653398-758f-4074-a9e6-706c45dee505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:19.928827 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1188/2391] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:19.941885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8d703-91f1-4ffc-a284-e69936a630ce admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:19.943795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e5f8d703-91f1-4ffc-a284-e69936a630ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:19.955462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:20.009934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8d703-91f1-4ffc-a284-e69936a630ce admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:20.013849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1d60c-91d6-4964-a1dd-4fa0ee40848f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:20.014478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1d60c-91d6-4964-a1dd-4fa0ee40848f service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:20.014760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1d60c-91d6-4964-a1dd-4fa0ee40848f service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:20.015110 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1d60c-91d6-4964-a1dd-4fa0ee40848f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:20.026945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1d60c-91d6-4964-a1dd-4fa0ee40848f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:20.028410 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1d60c-91d6-4964-a1dd-4fa0ee40848f service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:20.038982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1204/2392] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:20.108273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a653398-758f-4074-a9e6-706c45dee505 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:20.176686 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a653398-758f-4074-a9e6-706c45dee505 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:20.181936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:20.182584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:20.182898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:20.183181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] PATH_INFO: `/v3/users/303caf433510436b917db3e63cdda0e2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:20.184410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] RBAC: Authorizing `identity:delete_user(user_id=303caf433510436b917db3e63cdda0e2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:20.186382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:20.238450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303caf433510436b917db3e63cdda0e2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:20.239133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303caf433510436b917db3e63cdda0e2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:20.251535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303caf433510436b917db3e63cdda0e2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:20.256520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:20.267978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303caf433510436b917db3e63cdda0e2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:20.276038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303caf433510436b917db3e63cdda0e2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:20.286070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1189/2393] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:20 2022] DELETE /identity/v3/users/303caf433510436b917db3e63cdda0e2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:20.288142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d317-aaba-4871-8c56-d71914dc5bad None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:20.288431 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d317-aaba-4871-8c56-d71914dc5bad None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:20.288735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d317-aaba-4871-8c56-d71914dc5bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:20.317305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7f08d317-aaba-4871-8c56-d71914dc5bad None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:20.318524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7f08d317-aaba-4871-8c56-d71914dc5bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:20.400941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1205/2394] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:20.407558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:20.409046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:20.495506 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5035a94-b0ff-4230-a3fd-9caf8393e10d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:20.499121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddfd75-d5ec-4cee-b61f-96552a338929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:20.499735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddfd75-d5ec-4cee-b61f-96552a338929 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:20.500014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddfd75-d5ec-4cee-b61f-96552a338929 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:20.500903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddfd75-d5ec-4cee-b61f-96552a338929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:20.510158 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddfd75-d5ec-4cee-b61f-96552a338929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:20.511740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddfd75-d5ec-4cee-b61f-96552a338929 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:20.521064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1190/2395] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:20.792015 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d317-aaba-4871-8c56-d71914dc5bad None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:20.793482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7f08d317-aaba-4871-8c56-d71914dc5bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:20.806024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d317-aaba-4871-8c56-d71914dc5bad None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:20.810692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:20.811360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:20.811642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:20.811913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] PATH_INFO: `/v3/projects/e64ea134a7cb4983be82c17ddcc224d2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:20.813358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e64ea134a7cb4983be82c17ddcc224d2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:20.815161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:20.861987 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e64ea134a7cb4983be82c17ddcc224d2', 'request_id': 'req-e300b4aa-0e12-4f04-b409-42ca7b31088b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:20.867409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e64ea134a7cb4983be82c17ddcc224d2', 'request_id': 'req-e300b4aa-0e12-4f04-b409-42ca7b31088b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:20.873388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e64ea134a7cb4983be82c17ddcc224d2', 'request_id': 'req-e300b4aa-0e12-4f04-b409-42ca7b31088b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:20.884779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] The token cache is being invalidate because project e64ea134a7cb4983be82c17ddcc224d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:20.885066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:20.885688 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:20.887233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1206/2396] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:20 2022] DELETE /identity/v3/projects/e64ea134a7cb4983be82c17ddcc224d2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:20.892012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddfd75-d5ec-4cee-b61f-96552a338929 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:20.901688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddfd75-d5ec-4cee-b61f-96552a338929 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:20.960748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce8200-aa27-4762-874b-a1f092c18d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:20.961391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce8200-aa27-4762-874b-a1f092c18d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:20.961677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce8200-aa27-4762-874b-a1f092c18d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:20.961957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce8200-aa27-4762-874b-a1f092c18d64 admin admin] PATH_INFO: `/v3/users/df974507c2734fa5837fb26adcebcb52` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:20.963151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce8200-aa27-4762-874b-a1f092c18d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df974507c2734fa5837fb26adcebcb52)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:20.965071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce8200-aa27-4762-874b-a1f092c18d64 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:21.010053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cdce8200-aa27-4762-874b-a1f092c18d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df974507c2734fa5837fb26adcebcb52'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:21.010703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cdce8200-aa27-4762-874b-a1f092c18d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df974507c2734fa5837fb26adcebcb52'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:21.019653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cdce8200-aa27-4762-874b-a1f092c18d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df974507c2734fa5837fb26adcebcb52'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:21.033190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cdce8200-aa27-4762-874b-a1f092c18d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df974507c2734fa5837fb26adcebcb52'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:21.040410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cdce8200-aa27-4762-874b-a1f092c18d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df974507c2734fa5837fb26adcebcb52'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:21.041831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:21.047986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1191/2397] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:20 2022] DELETE /identity/v3/users/df974507c2734fa5837fb26adcebcb52 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:21.320332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:21.322036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:21.342417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300b4aa-0e12-4f04-b409-42ca7b31088b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:21.382762 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:21.413538 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:21.414176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:21.414458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:21.414737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] PATH_INFO: `/v3/projects/d50d0fe9270a44db8e3629f2fbe71111` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:21.415914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d50d0fe9270a44db8e3629f2fbe71111)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:21.417814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:21.462797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd50d0fe9270a44db8e3629f2fbe71111', 'request_id': 'req-63b4bb56-7555-4938-966c-33984c3ee5a8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:21.468205 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd50d0fe9270a44db8e3629f2fbe71111', 'request_id': 'req-63b4bb56-7555-4938-966c-33984c3ee5a8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:21.475077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd50d0fe9270a44db8e3629f2fbe71111', 'request_id': 'req-63b4bb56-7555-4938-966c-33984c3ee5a8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:21.488142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] The token cache is being invalidate because project d50d0fe9270a44db8e3629f2fbe71111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:21.488425 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:21.489045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-63b4bb56-7555-4938-966c-33984c3ee5a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:21.490612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1207/2398] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:21 2022] DELETE /identity/v3/projects/d50d0fe9270a44db8e3629f2fbe71111 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:21.495546 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37cb78-648f-4903-8c45-a7cb81de524f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:21.495932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37cb78-648f-4903-8c45-a7cb81de524f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:21.496095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37cb78-648f-4903-8c45-a7cb81de524f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:21.515981 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cb37cb78-648f-4903-8c45-a7cb81de524f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:21.517002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cb37cb78-648f-4903-8c45-a7cb81de524f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:21.595355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1192/2399] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:06:21.602276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d1064-12f6-4eed-81b8-913b203d93df None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:21.602584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d1064-12f6-4eed-81b8-913b203d93df None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:21.602899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d1064-12f6-4eed-81b8-913b203d93df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:21.624583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-636d1064-12f6-4eed-81b8-913b203d93df None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:21.626446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-636d1064-12f6-4eed-81b8-913b203d93df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:21.636085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1208/2400] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:21.640531 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37cb78-648f-4903-8c45-a7cb81de524f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:21.651390 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37cb78-648f-4903-8c45-a7cb81de524f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:21.655992 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1a947-4f26-48ea-b8b6-b8fefca11f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:21.656885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1a947-4f26-48ea-b8b6-b8fefca11f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:21.657007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1a947-4f26-48ea-b8b6-b8fefca11f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:21.657227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1a947-4f26-48ea-b8b6-b8fefca11f40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:21.657937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1a947-4f26-48ea-b8b6-b8fefca11f40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:06:21.658248 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1a947-4f26-48ea-b8b6-b8fefca11f40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:21.659886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1a947-4f26-48ea-b8b6-b8fefca11f40 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:21.673256 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1193/2401] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:06:21.677619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d1064-12f6-4eed-81b8-913b203d93df None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:21.687073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d1064-12f6-4eed-81b8-913b203d93df None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:21.692140 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a35ba-f8ba-4ca5-8e78-aace314dd560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:21.692755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a35ba-f8ba-4ca5-8e78-aace314dd560 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:21.693042 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a35ba-f8ba-4ca5-8e78-aace314dd560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:21.693320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a35ba-f8ba-4ca5-8e78-aace314dd560 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:21.693998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a35ba-f8ba-4ca5-8e78-aace314dd560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:21.696012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a35ba-f8ba-4ca5-8e78-aace314dd560 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:21.724265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1209/2402] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:21 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:21.728497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1a947-4f26-48ea-b8b6-b8fefca11f40 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:21.738343 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1a947-4f26-48ea-b8b6-b8fefca11f40 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:21.794357 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0a519-77fa-4a1d-b0ca-1a5a2dc995c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:21.795006 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0a519-77fa-4a1d-b0ca-1a5a2dc995c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:21.795302 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0a519-77fa-4a1d-b0ca-1a5a2dc995c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:21.795583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0a519-77fa-4a1d-b0ca-1a5a2dc995c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:21.796261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0a519-77fa-4a1d-b0ca-1a5a2dc995c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:21.798862 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0a519-77fa-4a1d-b0ca-1a5a2dc995c8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:21.820488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1194/2403] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:21 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:21.825482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a35ba-f8ba-4ca5-8e78-aace314dd560 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:21.836516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a35ba-f8ba-4ca5-8e78-aace314dd560 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:21.841580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd169398-c791-456c-9e0a-10188875a0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:21.842356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd169398-c791-456c-9e0a-10188875a0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:21.842454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd169398-c791-456c-9e0a-10188875a0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:21.842725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd169398-c791-456c-9e0a-10188875a0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:21.843462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd169398-c791-456c-9e0a-10188875a0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:21.845250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd169398-c791-456c-9e0a-10188875a0d0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:21.855584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1210/2404] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:21.860358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0a519-77fa-4a1d-b0ca-1a5a2dc995c8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:21.870977 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0a519-77fa-4a1d-b0ca-1a5a2dc995c8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:21.876498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3faa0-ab38-466a-aa17-b8bbf9513653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:21.877119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3faa0-ab38-466a-aa17-b8bbf9513653 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:21.877405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3faa0-ab38-466a-aa17-b8bbf9513653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:21.877682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3faa0-ab38-466a-aa17-b8bbf9513653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:21.878391 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3faa0-ab38-466a-aa17-b8bbf9513653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:21.880101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3faa0-ab38-466a-aa17-b8bbf9513653 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:21.892687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1195/2405] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:21.896827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd169398-c791-456c-9e0a-10188875a0d0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:21.907026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd169398-c791-456c-9e0a-10188875a0d0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:21.912006 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421b16d-ae83-4cfa-863d-1ffa2416ddfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:21.912694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b16d-ae83-4cfa-863d-1ffa2416ddfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:21.912975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b16d-ae83-4cfa-863d-1ffa2416ddfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:21.913248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b16d-ae83-4cfa-863d-1ffa2416ddfc admin admin] PATH_INFO: `/v3/projects/59987f0406db4cd18afef7162cb36b66/users/7c9446e453bf4104b9e702c1fbbfcb25/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:21.928245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421b16d-ae83-4cfa-863d-1ffa2416ddfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=59987f0406db4cd18afef7162cb36b66, user_id=7c9446e453bf4104b9e702c1fbbfcb25, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:21.930897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421b16d-ae83-4cfa-863d-1ffa2416ddfc admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:21.944383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1211/2406] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:21 2022] PUT /identity/v3/projects/59987f0406db4cd18afef7162cb36b66/users/7c9446e453bf4104b9e702c1fbbfcb25/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:21.955999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535eedd-47de-486b-8cec-c4b7e4404ad2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:21.956286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535eedd-47de-486b-8cec-c4b7e4404ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:21.956584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535eedd-47de-486b-8cec-c4b7e4404ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:21.983128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1535eedd-47de-486b-8cec-c4b7e4404ad2 None None] MFA Rules not processed for user `7c9446e453bf4104b9e702c1fbbfcb25`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:21.984326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1535eedd-47de-486b-8cec-c4b7e4404ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:22.038353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:22.050201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1196/2407] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:22.054650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421b16d-ae83-4cfa-863d-1ffa2416ddfc admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:22.116142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421b16d-ae83-4cfa-863d-1ffa2416ddfc admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:22.121040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2d205-fbbf-4e92-ace3-50ef22d6705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:22.121646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2d205-fbbf-4e92-ace3-50ef22d6705e admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.121925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2d205-fbbf-4e92-ace3-50ef22d6705e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.122194 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2d205-fbbf-4e92-ace3-50ef22d6705e admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.122885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2d205-fbbf-4e92-ace3-50ef22d6705e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:22.124796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2d205-fbbf-4e92-ace3-50ef22d6705e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:22.129318 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:22.152643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1212/2408] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:22 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:22.157168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535eedd-47de-486b-8cec-c4b7e4404ad2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:22.167913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535eedd-47de-486b-8cec-c4b7e4404ad2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:22.222875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79275f0-cc87-4b45-b9c1-3a625a3321b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:22.223482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79275f0-cc87-4b45-b9c1-3a625a3321b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.223758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79275f0-cc87-4b45-b9c1-3a625a3321b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.224029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79275f0-cc87-4b45-b9c1-3a625a3321b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.224691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79275f0-cc87-4b45-b9c1-3a625a3321b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:22.226603 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79275f0-cc87-4b45-b9c1-3a625a3321b3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:22.247281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1197/2409] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:22 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:22.251900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2d205-fbbf-4e92-ace3-50ef22d6705e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:22.262212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2d205-fbbf-4e92-ace3-50ef22d6705e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:22.267566 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e9db9-4841-494e-9aa8-775740522ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:22.268173 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e9db9-4841-494e-9aa8-775740522ecb admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.268457 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e9db9-4841-494e-9aa8-775740522ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.268733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e9db9-4841-494e-9aa8-775740522ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.269430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e9db9-4841-494e-9aa8-775740522ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:22.271070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e9db9-4841-494e-9aa8-775740522ecb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:22.280863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1213/2410] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:22.284746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79275f0-cc87-4b45-b9c1-3a625a3321b3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:22.294930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79275f0-cc87-4b45-b9c1-3a625a3321b3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:22.299967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e07232-d289-460f-9229-e152a012790b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:22.300577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e07232-d289-460f-9229-e152a012790b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.300863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e07232-d289-460f-9229-e152a012790b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.301142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e07232-d289-460f-9229-e152a012790b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.301839 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e07232-d289-460f-9229-e152a012790b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:22.303545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e07232-d289-460f-9229-e152a012790b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:22.314100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1198/2411] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:22.318132 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e9db9-4841-494e-9aa8-775740522ecb admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:22.328041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e9db9-4841-494e-9aa8-775740522ecb admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:22.332969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53aad03-c75d-4415-987f-ed6bd55468a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:22.333630 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aad03-c75d-4415-987f-ed6bd55468a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.333913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aad03-c75d-4415-987f-ed6bd55468a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.334191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aad03-c75d-4415-987f-ed6bd55468a8 admin admin] PATH_INFO: `/v3/projects/2e303719c87f4a63afbea82082847c51/users/f374dc4081dd49ca8a5d4ed29a92c742/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.350201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53aad03-c75d-4415-987f-ed6bd55468a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e303719c87f4a63afbea82082847c51, user_id=f374dc4081dd49ca8a5d4ed29a92c742, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:22.352891 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53aad03-c75d-4415-987f-ed6bd55468a8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:22.368163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1214/2412] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:22 2022] PUT /identity/v3/projects/2e303719c87f4a63afbea82082847c51/users/f374dc4081dd49ca8a5d4ed29a92c742/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:22.378776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dbb59-933b-4c08-8357-1fb595c4d21c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.379091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dbb59-933b-4c08-8357-1fb595c4d21c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.379367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dbb59-933b-4c08-8357-1fb595c4d21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.411766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0b4dbb59-933b-4c08-8357-1fb595c4d21c None None] MFA Rules not processed for user `f374dc4081dd49ca8a5d4ed29a92c742`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:22.412990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0b4dbb59-933b-4c08-8357-1fb595c4d21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:22.476167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1199/2413] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:22.480715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53aad03-c75d-4415-987f-ed6bd55468a8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:22.542496 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53aad03-c75d-4415-987f-ed6bd55468a8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:22.547726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d06c3-8101-40c2-899e-a8586e79e2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:22.548386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d06c3-8101-40c2-899e-a8586e79e2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.548665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d06c3-8101-40c2-899e-a8586e79e2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.548937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d06c3-8101-40c2-899e-a8586e79e2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.549608 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d06c3-8101-40c2-899e-a8586e79e2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:22.551545 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d06c3-8101-40c2-899e-a8586e79e2a4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:22.580723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1215/2414] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:22 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:22.585280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dbb59-933b-4c08-8357-1fb595c4d21c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:22.595986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dbb59-933b-4c08-8357-1fb595c4d21c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:22.655344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f77c79-2d74-4905-bbf6-20622ec26258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:22.655962 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f77c79-2d74-4905-bbf6-20622ec26258 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.656247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f77c79-2d74-4905-bbf6-20622ec26258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.656766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f77c79-2d74-4905-bbf6-20622ec26258 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.657448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f77c79-2d74-4905-bbf6-20622ec26258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:22.659367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f77c79-2d74-4905-bbf6-20622ec26258 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:22.668785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:22.686102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1200/2415] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:22 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:22.687233 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d06c3-8101-40c2-899e-a8586e79e2a4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:22.701049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d06c3-8101-40c2-899e-a8586e79e2a4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:22.707168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ca5c1-6236-4749-aae5-3425837fa1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:22.707790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ca5c1-6236-4749-aae5-3425837fa1ee admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.708073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ca5c1-6236-4749-aae5-3425837fa1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.708348 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ca5c1-6236-4749-aae5-3425837fa1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.709045 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ca5c1-6236-4749-aae5-3425837fa1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:22.710638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ca5c1-6236-4749-aae5-3425837fa1ee admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:22.726842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1216/2416] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:22.727983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f77c79-2d74-4905-bbf6-20622ec26258 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:22.739975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f77c79-2d74-4905-bbf6-20622ec26258 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:22.744746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb4944-69db-48a2-9924-e9c276833fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:22.745487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb4944-69db-48a2-9924-e9c276833fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.745838 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb4944-69db-48a2-9924-e9c276833fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.746613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb4944-69db-48a2-9924-e9c276833fe5 admin admin] PATH_INFO: `/v3/projects/7bc041424477422484938183d5d967eb/users/8774bb567e854744b5086e28eb439256/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.764232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb4944-69db-48a2-9924-e9c276833fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc041424477422484938183d5d967eb, user_id=8774bb567e854744b5086e28eb439256, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:22.767589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb4944-69db-48a2-9924-e9c276833fe5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:22.784171 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1201/2417] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:22 2022] PUT /identity/v3/projects/7bc041424477422484938183d5d967eb/users/8774bb567e854744b5086e28eb439256/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:22.788403 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ca5c1-6236-4749-aae5-3425837fa1ee admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:22.817599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ca5c1-6236-4749-aae5-3425837fa1ee admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:22.889199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e991b-79b4-4721-a867-baf907d479e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:22.889850 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e991b-79b4-4721-a867-baf907d479e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.890012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e991b-79b4-4721-a867-baf907d479e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.890290 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e991b-79b4-4721-a867-baf907d479e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.891738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e991b-79b4-4721-a867-baf907d479e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:22.893343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e991b-79b4-4721-a867-baf907d479e2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:22.903341 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1217/2418] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:22.907694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb4944-69db-48a2-9924-e9c276833fe5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:22.918111 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb4944-69db-48a2-9924-e9c276833fe5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:22.923295 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b982b6-0704-4dfd-929e-fd7a1d8169c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:22.924043 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b982b6-0704-4dfd-929e-fd7a1d8169c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.924324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b982b6-0704-4dfd-929e-fd7a1d8169c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.924598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b982b6-0704-4dfd-929e-fd7a1d8169c8 admin admin] PATH_INFO: `/v3/projects/7bc041424477422484938183d5d967eb/users/8774bb567e854744b5086e28eb439256/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.926583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b982b6-0704-4dfd-929e-fd7a1d8169c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc041424477422484938183d5d967eb, user_id=8774bb567e854744b5086e28eb439256, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:22.929409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b982b6-0704-4dfd-929e-fd7a1d8169c8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:22.940966 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1202/2419] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:22 2022] PUT /identity/v3/projects/7bc041424477422484938183d5d967eb/users/8774bb567e854744b5086e28eb439256/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:22.951313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8351b-dc93-442a-a791-a765811b957c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:22.951621 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8351b-dc93-442a-a791-a765811b957c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:22.951894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8351b-dc93-442a-a791-a765811b957c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:22.980159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ebf8351b-dc93-442a-a791-a765811b957c None None] MFA Rules not processed for user `8774bb567e854744b5086e28eb439256`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:22.982943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ebf8351b-dc93-442a-a791-a765811b957c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:22.990421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:23.051017 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1218/2420] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:23.061936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b982b6-0704-4dfd-929e-fd7a1d8169c8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:23.064339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-29b982b6-0704-4dfd-929e-fd7a1d8169c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:23.120423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b982b6-0704-4dfd-929e-fd7a1d8169c8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:23.124023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb4fc7-32e3-4dcf-8820-b55400c0dd82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:23.124629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb4fc7-32e3-4dcf-8820-b55400c0dd82 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:23.124909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb4fc7-32e3-4dcf-8820-b55400c0dd82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:23.125836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb4fc7-32e3-4dcf-8820-b55400c0dd82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:23.135144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb4fc7-32e3-4dcf-8820-b55400c0dd82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:23.136566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb4fc7-32e3-4dcf-8820-b55400c0dd82 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:23.146655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1203/2421] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:23.187487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8351b-dc93-442a-a791-a765811b957c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:23.189473 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ebf8351b-dc93-442a-a791-a765811b957c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:23.201176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8351b-dc93-442a-a791-a765811b957c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:23.204805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb21d34-f61d-4755-b195-97a2c0364414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:23.205403 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb21d34-f61d-4755-b195-97a2c0364414 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:23.205682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb21d34-f61d-4755-b195-97a2c0364414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:23.205952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb21d34-f61d-4755-b195-97a2c0364414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:23.267767 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb21d34-f61d-4755-b195-97a2c0364414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:23.269207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb21d34-f61d-4755-b195-97a2c0364414 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:23.277927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1219/2422] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:26.198465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb4fc7-32e3-4dcf-8820-b55400c0dd82 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:26.276486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb4fc7-32e3-4dcf-8820-b55400c0dd82 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:26.287769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:26.288324 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:26.288695 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:26.288980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] PATH_INFO: `/v3/users/7c9446e453bf4104b9e702c1fbbfcb25` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:26.291338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c9446e453bf4104b9e702c1fbbfcb25)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:26.292853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:26.379880 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c9446e453bf4104b9e702c1fbbfcb25'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:26.381059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c9446e453bf4104b9e702c1fbbfcb25'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:26.390497 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c9446e453bf4104b9e702c1fbbfcb25'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:26.399871 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:26.409984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c9446e453bf4104b9e702c1fbbfcb25'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:26.430766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c9446e453bf4104b9e702c1fbbfcb25'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:26.438112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1204/2423] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:26 2022] DELETE /identity/v3/users/7c9446e453bf4104b9e702c1fbbfcb25 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:26.442910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651049b-925a-484b-9a42-15fa2315136b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:26.443415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651049b-925a-484b-9a42-15fa2315136b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:26.443691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651049b-925a-484b-9a42-15fa2315136b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:26.472368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a651049b-925a-484b-9a42-15fa2315136b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:26.473732 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a651049b-925a-484b-9a42-15fa2315136b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:26.558856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1220/2424] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:26.568660 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:26.570357 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:26.667207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c55e23-b9e3-44bc-8a48-648c8c7f8f34 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:26.671262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bb182-15ab-43ec-9eda-5d2f370adce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:26.671945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bb182-15ab-43ec-9eda-5d2f370adce9 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:26.672245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bb182-15ab-43ec-9eda-5d2f370adce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:26.672510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bb182-15ab-43ec-9eda-5d2f370adce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:26.682456 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bb182-15ab-43ec-9eda-5d2f370adce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:26.683843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bb182-15ab-43ec-9eda-5d2f370adce9 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:26.694639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1205/2425] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:27.086443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651049b-925a-484b-9a42-15fa2315136b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:27.088102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a651049b-925a-484b-9a42-15fa2315136b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:27.101465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651049b-925a-484b-9a42-15fa2315136b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:27.105996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:27.106667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:27.106982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:27.107259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] PATH_INFO: `/v3/projects/59987f0406db4cd18afef7162cb36b66` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:27.108557 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=59987f0406db4cd18afef7162cb36b66)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:27.111209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:27.157860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:27.166546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59987f0406db4cd18afef7162cb36b66', 'request_id': 'req-7428c875-6ecd-446e-90c2-f614c96502cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.173954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59987f0406db4cd18afef7162cb36b66', 'request_id': 'req-7428c875-6ecd-446e-90c2-f614c96502cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.183303 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59987f0406db4cd18afef7162cb36b66', 'request_id': 'req-7428c875-6ecd-446e-90c2-f614c96502cc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.198011 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] The token cache is being invalidate because project 59987f0406db4cd18afef7162cb36b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:27.198319 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.199057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.200585 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1221/2426] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:27 2022] DELETE /identity/v3/projects/59987f0406db4cd18afef7162cb36b66 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:27.205532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bb182-15ab-43ec-9eda-5d2f370adce9 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:27.215145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bb182-15ab-43ec-9eda-5d2f370adce9 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:27.286012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:27.286998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:27.287221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:27.293623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] PATH_INFO: `/v3/users/f374dc4081dd49ca8a5d4ed29a92c742` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:27.295257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f374dc4081dd49ca8a5d4ed29a92c742)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:27.298550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:27.351856 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f374dc4081dd49ca8a5d4ed29a92c742'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.352522 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f374dc4081dd49ca8a5d4ed29a92c742'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.361512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f374dc4081dd49ca8a5d4ed29a92c742'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.373985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f374dc4081dd49ca8a5d4ed29a92c742'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.381846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f374dc4081dd49ca8a5d4ed29a92c742'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.383381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:27.389387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1206/2427] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:27 2022] DELETE /identity/v3/users/f374dc4081dd49ca8a5d4ed29a92c742 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:27.774827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:27.776367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:27.790127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428c875-6ecd-446e-90c2-f614c96502cc admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:27.857378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:27.858022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:27.858305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:27.858582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] PATH_INFO: `/v3/projects/2e303719c87f4a63afbea82082847c51` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:27.861084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e303719c87f4a63afbea82082847c51)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:27.863156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:27.909760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e303719c87f4a63afbea82082847c51', 'request_id': 'req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.915534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e303719c87f4a63afbea82082847c51', 'request_id': 'req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.922385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e303719c87f4a63afbea82082847c51', 'request_id': 'req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.935163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] The token cache is being invalidate because project 2e303719c87f4a63afbea82082847c51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:27.935435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.939739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:27.941225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1222/2428] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:27 2022] DELETE /identity/v3/projects/2e303719c87f4a63afbea82082847c51 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:27.945235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:27.947345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:27.961020 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43dd2f-7fc4-4a01-91ca-6e5d435a74ed admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:28.019851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddab9e5-1839-4114-aa70-a96e761f9cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:28.020501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddab9e5-1839-4114-aa70-a96e761f9cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:28.021398 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddab9e5-1839-4114-aa70-a96e761f9cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:28.021687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddab9e5-1839-4114-aa70-a96e761f9cb7 admin admin] PATH_INFO: `/v3/users/8774bb567e854744b5086e28eb439256` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:28.023071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddab9e5-1839-4114-aa70-a96e761f9cb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8774bb567e854744b5086e28eb439256)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:28.024978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddab9e5-1839-4114-aa70-a96e761f9cb7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:28.071757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cddab9e5-1839-4114-aa70-a96e761f9cb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8774bb567e854744b5086e28eb439256'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:28.072467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cddab9e5-1839-4114-aa70-a96e761f9cb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8774bb567e854744b5086e28eb439256'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:28.081196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cddab9e5-1839-4114-aa70-a96e761f9cb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8774bb567e854744b5086e28eb439256'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:28.094301 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cddab9e5-1839-4114-aa70-a96e761f9cb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8774bb567e854744b5086e28eb439256'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:28.101316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cddab9e5-1839-4114-aa70-a96e761f9cb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8774bb567e854744b5086e28eb439256'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:28.107538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1207/2429] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:27 2022] DELETE /identity/v3/users/8774bb567e854744b5086e28eb439256 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:28.402701 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:28.404379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:28.426468 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad06894-d4a9-4777-9d3f-7ac01d73a5e4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:28.496839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:28.511428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:28.511428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:28.511428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:28.511428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] PATH_INFO: `/v3/projects/7bc041424477422484938183d5d967eb` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:28.511857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bc041424477422484938183d5d967eb)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:28.511857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:28.576927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bc041424477422484938183d5d967eb', 'request_id': 'req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:28.586261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bc041424477422484938183d5d967eb', 'request_id': 'req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:28.595273 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bc041424477422484938183d5d967eb', 'request_id': 'req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:28.613998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] The token cache is being invalidate because project 7bc041424477422484938183d5d967eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:28.614280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:28.614945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cc693fd2-a39f-4a4d-aec3-6617e8b06c40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:28.616493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1223/2430] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:28 2022] DELETE /identity/v3/projects/7bc041424477422484938183d5d967eb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:28.622479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731dc75-b2e4-4442-9123-97007e891287 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:28.623000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731dc75-b2e4-4442-9123-97007e891287 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:28.623692 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731dc75-b2e4-4442-9123-97007e891287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:28.644659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c731dc75-b2e4-4442-9123-97007e891287 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:28.645885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c731dc75-b2e4-4442-9123-97007e891287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:28.712122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1208/2431] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:06:28.721349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38bc08-e210-49dc-9a28-67d91d76ffd3 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:28.721734 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38bc08-e210-49dc-9a28-67d91d76ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:28.722025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38bc08-e210-49dc-9a28-67d91d76ffd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:28.743378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-de38bc08-e210-49dc-9a28-67d91d76ffd3 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:28.744672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-de38bc08-e210-49dc-9a28-67d91d76ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:28.755684 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1224/2432] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:06:28.760299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731dc75-b2e4-4442-9123-97007e891287 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:28.772947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731dc75-b2e4-4442-9123-97007e891287 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:28.778054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c02b-c10c-40a6-93d9-61b25c3da148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:28.778845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c02b-c10c-40a6-93d9-61b25c3da148 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:28.779258 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c02b-c10c-40a6-93d9-61b25c3da148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:28.779576 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c02b-c10c-40a6-93d9-61b25c3da148 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:28.780383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0c02b-c10c-40a6-93d9-61b25c3da148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:06:28.780789 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0c02b-c10c-40a6-93d9-61b25c3da148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:28.783440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0c02b-c10c-40a6-93d9-61b25c3da148 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:28.796629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1209/2433] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:06:28.801378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38bc08-e210-49dc-9a28-67d91d76ffd3 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:28.810598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38bc08-e210-49dc-9a28-67d91d76ffd3 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:28.816599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3892e3-1954-4e7b-9869-9e09008f34ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:28.817207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3892e3-1954-4e7b-9869-9e09008f34ba admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:28.817487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3892e3-1954-4e7b-9869-9e09008f34ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:28.817794 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3892e3-1954-4e7b-9869-9e09008f34ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:28.818451 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3892e3-1954-4e7b-9869-9e09008f34ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:28.820373 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3892e3-1954-4e7b-9869-9e09008f34ba admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:28.849224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1225/2434] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:28 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:28.853664 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c02b-c10c-40a6-93d9-61b25c3da148 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:28.860501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:28.865202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c02b-c10c-40a6-93d9-61b25c3da148 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:28.927342 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fbcdc-05e3-42f6-9aba-98210c5e4f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:28.927961 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fbcdc-05e3-42f6-9aba-98210c5e4f86 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:28.928247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fbcdc-05e3-42f6-9aba-98210c5e4f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:28.928525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fbcdc-05e3-42f6-9aba-98210c5e4f86 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:28.929194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fbcdc-05e3-42f6-9aba-98210c5e4f86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:28.931147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fbcdc-05e3-42f6-9aba-98210c5e4f86 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:28.959527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1210/2435] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:28 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:28.960786 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3892e3-1954-4e7b-9869-9e09008f34ba admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:28.971756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3892e3-1954-4e7b-9869-9e09008f34ba admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:28.976378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed1941-7182-4505-9c33-7721b2e73bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:28.977091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1941-7182-4505-9c33-7721b2e73bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:28.977377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1941-7182-4505-9c33-7721b2e73bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:28.977650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1941-7182-4505-9c33-7721b2e73bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:28.978344 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed1941-7182-4505-9c33-7721b2e73bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:28.979959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed1941-7182-4505-9c33-7721b2e73bf8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:28.990329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1226/2436] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:28.994479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fbcdc-05e3-42f6-9aba-98210c5e4f86 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:29.005646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fbcdc-05e3-42f6-9aba-98210c5e4f86 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:29.010494 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20875e3b-f3b7-4912-802e-b568d5fdd636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:29.011148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20875e3b-f3b7-4912-802e-b568d5fdd636 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:29.011430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20875e3b-f3b7-4912-802e-b568d5fdd636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:29.011701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20875e3b-f3b7-4912-802e-b568d5fdd636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:29.012390 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20875e3b-f3b7-4912-802e-b568d5fdd636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:29.013973 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20875e3b-f3b7-4912-802e-b568d5fdd636 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:29.025491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1211/2437] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:29.029690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed1941-7182-4505-9c33-7721b2e73bf8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:29.040402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed1941-7182-4505-9c33-7721b2e73bf8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:29.045068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c41fa8-4891-4d99-950a-1f723926aeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:29.045725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c41fa8-4891-4d99-950a-1f723926aeae admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:29.046008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c41fa8-4891-4d99-950a-1f723926aeae admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:29.046284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c41fa8-4891-4d99-950a-1f723926aeae admin admin] PATH_INFO: `/v3/projects/a56a2f45a3ba42169d64085f73ffe84e/users/7ac9338d00ac40e4afbf299505793d45/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:29.062208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c41fa8-4891-4d99-950a-1f723926aeae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56a2f45a3ba42169d64085f73ffe84e, user_id=7ac9338d00ac40e4afbf299505793d45, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:29.065804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c41fa8-4891-4d99-950a-1f723926aeae admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:29.079192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1227/2438] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:29 2022] PUT /identity/v3/projects/a56a2f45a3ba42169d64085f73ffe84e/users/7ac9338d00ac40e4afbf299505793d45/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:29.090246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5c1a5-ddd8-4863-88ec-40b6b6d8e511 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:29.090538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5c1a5-ddd8-4863-88ec-40b6b6d8e511 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:29.090844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5c1a5-ddd8-4863-88ec-40b6b6d8e511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:29.121181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b2e5c1a5-ddd8-4863-88ec-40b6b6d8e511 None None] MFA Rules not processed for user `7ac9338d00ac40e4afbf299505793d45`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:29.122654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b2e5c1a5-ddd8-4863-88ec-40b6b6d8e511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:29.194409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1212/2439] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:29.198971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c41fa8-4891-4d99-950a-1f723926aeae admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:29.263167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c41fa8-4891-4d99-950a-1f723926aeae admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:29.268127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe0b74-e7c2-4668-8dd5-16a9e2a21fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:29.268853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe0b74-e7c2-4668-8dd5-16a9e2a21fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:29.269134 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe0b74-e7c2-4668-8dd5-16a9e2a21fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:29.269412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe0b74-e7c2-4668-8dd5-16a9e2a21fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:29.270079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe0b74-e7c2-4668-8dd5-16a9e2a21fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:29.271997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe0b74-e7c2-4668-8dd5-16a9e2a21fb9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:29.299892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1228/2440] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:29 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:06:29.303574 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5c1a5-ddd8-4863-88ec-40b6b6d8e511 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:29.315468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5c1a5-ddd8-4863-88ec-40b6b6d8e511 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:29.379741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cac2c-648b-4174-9539-3cc7fd1c44a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:29.380367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cac2c-648b-4174-9539-3cc7fd1c44a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:29.380674 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cac2c-648b-4174-9539-3cc7fd1c44a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:29.380945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cac2c-648b-4174-9539-3cc7fd1c44a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:29.381645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cac2c-648b-4174-9539-3cc7fd1c44a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:29.383597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cac2c-648b-4174-9539-3cc7fd1c44a6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:29.407049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1213/2441] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:29 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:29.411124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe0b74-e7c2-4668-8dd5-16a9e2a21fb9 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:29.421743 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe0b74-e7c2-4668-8dd5-16a9e2a21fb9 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:29.427245 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c718c3-c03c-4525-bdab-51be69a2226e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:29.427623 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c718c3-c03c-4525-bdab-51be69a2226e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:29.427905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c718c3-c03c-4525-bdab-51be69a2226e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:29.428179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c718c3-c03c-4525-bdab-51be69a2226e admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:29.428875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c718c3-c03c-4525-bdab-51be69a2226e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:29.430478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c718c3-c03c-4525-bdab-51be69a2226e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:29.432689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:29.443335 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1229/2442] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:29.450906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cac2c-648b-4174-9539-3cc7fd1c44a6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:29.457211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:29.463538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cac2c-648b-4174-9539-3cc7fd1c44a6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:29.470321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027e927-1734-4004-a839-c5b85ec0d377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:29.471026 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027e927-1734-4004-a839-c5b85ec0d377 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:29.471333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027e927-1734-4004-a839-c5b85ec0d377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:29.471608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027e927-1734-4004-a839-c5b85ec0d377 admin admin] PATH_INFO: `/v3/projects/9eff3112495d4fd98391d9d392d75125/users/7d6075ad911440ee89012c0ee4dde156/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:29.488855 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027e927-1734-4004-a839-c5b85ec0d377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eff3112495d4fd98391d9d392d75125, user_id=7d6075ad911440ee89012c0ee4dde156, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:29.491655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027e927-1734-4004-a839-c5b85ec0d377 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:29.506178 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1214/2443] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:29 2022] PUT /identity/v3/projects/9eff3112495d4fd98391d9d392d75125/users/7d6075ad911440ee89012c0ee4dde156/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:29.510074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c718c3-c03c-4525-bdab-51be69a2226e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:29.520775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c718c3-c03c-4525-bdab-51be69a2226e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:29.608632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e88e7a-d00a-42f7-85ed-bc240c91a822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:29.609917 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e88e7a-d00a-42f7-85ed-bc240c91a822 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:29.610733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e88e7a-d00a-42f7-85ed-bc240c91a822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:29.610733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e88e7a-d00a-42f7-85ed-bc240c91a822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:29.611232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e88e7a-d00a-42f7-85ed-bc240c91a822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:29.613320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e88e7a-d00a-42f7-85ed-bc240c91a822 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:29.630918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1230/2444] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:29.632326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027e927-1734-4004-a839-c5b85ec0d377 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:29.644769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027e927-1734-4004-a839-c5b85ec0d377 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:29.669635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fd2da-4e58-4784-8f8d-9dcec06f4001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:29.671495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fd2da-4e58-4784-8f8d-9dcec06f4001 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:29.672560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fd2da-4e58-4784-8f8d-9dcec06f4001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:29.672878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fd2da-4e58-4784-8f8d-9dcec06f4001 admin admin] PATH_INFO: `/v3/projects/9eff3112495d4fd98391d9d392d75125/users/7d6075ad911440ee89012c0ee4dde156/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:29.684327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fd2da-4e58-4784-8f8d-9dcec06f4001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eff3112495d4fd98391d9d392d75125, user_id=7d6075ad911440ee89012c0ee4dde156, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:29.689386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fd2da-4e58-4784-8f8d-9dcec06f4001 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:29.722947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1215/2445] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:29 2022] PUT /identity/v3/projects/9eff3112495d4fd98391d9d392d75125/users/7d6075ad911440ee89012c0ee4dde156/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:29.728119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08948f7-0595-4f67-8dc0-1fec5a490913 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:29.728660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08948f7-0595-4f67-8dc0-1fec5a490913 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:29.729190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08948f7-0595-4f67-8dc0-1fec5a490913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:29.775401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b08948f7-0595-4f67-8dc0-1fec5a490913 None None] MFA Rules not processed for user `7d6075ad911440ee89012c0ee4dde156`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:29.777423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b08948f7-0595-4f67-8dc0-1fec5a490913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:29.910308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1231/2446] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:29.938424 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fd2da-4e58-4784-8f8d-9dcec06f4001 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:29.940661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-885fd2da-4e58-4784-8f8d-9dcec06f4001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:30.027986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fd2da-4e58-4784-8f8d-9dcec06f4001 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:30.033460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3225f-3482-4aba-94cd-a48ebfd9d244 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:30.034316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3225f-3482-4aba-94cd-a48ebfd9d244 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:30.034932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3225f-3482-4aba-94cd-a48ebfd9d244 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:30.035280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3225f-3482-4aba-94cd-a48ebfd9d244 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:30.048558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3225f-3482-4aba-94cd-a48ebfd9d244 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:30.050549 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3225f-3482-4aba-94cd-a48ebfd9d244 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:30.063804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1216/2447] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:30.269796 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08948f7-0595-4f67-8dc0-1fec5a490913 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:30.271789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b08948f7-0595-4f67-8dc0-1fec5a490913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:30.382726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08948f7-0595-4f67-8dc0-1fec5a490913 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:30.388181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3f2f2-3dfc-4319-bf23-908f5e8d77ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:30.388882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3f2f2-3dfc-4319-bf23-908f5e8d77ec service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:30.389124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3f2f2-3dfc-4319-bf23-908f5e8d77ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:30.389407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3f2f2-3dfc-4319-bf23-908f5e8d77ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:30.393506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-80c3f2f2-3dfc-4319-bf23-908f5e8d77ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:30.467969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:30.522293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3f2f2-3dfc-4319-bf23-908f5e8d77ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:30.524272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3f2f2-3dfc-4319-bf23-908f5e8d77ec service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:30.553793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1232/2448] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:30.863452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3225f-3482-4aba-94cd-a48ebfd9d244 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:30.876651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3225f-3482-4aba-94cd-a48ebfd9d244 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:30.882127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:30.882768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:30.883080 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:30.883734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] PATH_INFO: `/v3/users/7ac9338d00ac40e4afbf299505793d45` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:30.884649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ac9338d00ac40e4afbf299505793d45)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:30.886565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:30.952103 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ac9338d00ac40e4afbf299505793d45'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:30.953549 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ac9338d00ac40e4afbf299505793d45'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:30.962112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ac9338d00ac40e4afbf299505793d45'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:30.979702 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ac9338d00ac40e4afbf299505793d45'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:30.987156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ac9338d00ac40e4afbf299505793d45'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:30.993325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1217/2449] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:30 2022] DELETE /identity/v3/users/7ac9338d00ac40e4afbf299505793d45 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:30.997891 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bb3c8-3521-40a0-9b72-560be1693fa2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:30.998368 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bb3c8-3521-40a0-9b72-560be1693fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:30.998609 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bb3c8-3521-40a0-9b72-560be1693fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:31.020172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-845bb3c8-3521-40a0-9b72-560be1693fa2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:31.021492 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-845bb3c8-3521-40a0-9b72-560be1693fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:31.085293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1233/2450] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:31.095378 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:31.096779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:31.141900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:31.157847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31568845-ae7f-4efd-8740-3288754778fb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:31.162231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c7027-4d42-409a-b518-2d332c061b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:31.163019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c7027-4d42-409a-b518-2d332c061b2e service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:31.163424 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c7027-4d42-409a-b518-2d332c061b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:31.163709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c7027-4d42-409a-b518-2d332c061b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:31.173359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c7027-4d42-409a-b518-2d332c061b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:31.174809 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c7027-4d42-409a-b518-2d332c061b2e service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:31.184484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1218/2451] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:31.761241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bb3c8-3521-40a0-9b72-560be1693fa2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:31.762723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-845bb3c8-3521-40a0-9b72-560be1693fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:31.779605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bb3c8-3521-40a0-9b72-560be1693fa2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:31.785812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:31.786823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:31.787176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:31.787523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] PATH_INFO: `/v3/projects/a56a2f45a3ba42169d64085f73ffe84e` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:31.788882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a56a2f45a3ba42169d64085f73ffe84e)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:31.791208 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:31.845659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a56a2f45a3ba42169d64085f73ffe84e', 'request_id': 'req-10806f7f-3b6b-40e8-8e06-b0228e2a8417'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:31.852149 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a56a2f45a3ba42169d64085f73ffe84e', 'request_id': 'req-10806f7f-3b6b-40e8-8e06-b0228e2a8417'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:31.858614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a56a2f45a3ba42169d64085f73ffe84e', 'request_id': 'req-10806f7f-3b6b-40e8-8e06-b0228e2a8417'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:31.872835 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] The token cache is being invalidate because project a56a2f45a3ba42169d64085f73ffe84e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:31.873123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:31.875475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:31.876953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1234/2452] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:31 2022] DELETE /identity/v3/projects/a56a2f45a3ba42169d64085f73ffe84e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:31.880804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c7027-4d42-409a-b518-2d332c061b2e service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:31.891320 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c7027-4d42-409a-b518-2d332c061b2e service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:31.949854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2bd93-b748-40b0-8374-4be4852831c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:31.950521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2bd93-b748-40b0-8374-4be4852831c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:31.950819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2bd93-b748-40b0-8374-4be4852831c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:31.951094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2bd93-b748-40b0-8374-4be4852831c1 admin admin] PATH_INFO: `/v3/users/7d6075ad911440ee89012c0ee4dde156` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:31.952246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2bd93-b748-40b0-8374-4be4852831c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6075ad911440ee89012c0ee4dde156)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:31.955086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2bd93-b748-40b0-8374-4be4852831c1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:32.003980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ab2bd93-b748-40b0-8374-4be4852831c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6075ad911440ee89012c0ee4dde156'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:32.004922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ab2bd93-b748-40b0-8374-4be4852831c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6075ad911440ee89012c0ee4dde156'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:32.013448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ab2bd93-b748-40b0-8374-4be4852831c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6075ad911440ee89012c0ee4dde156'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:32.026751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ab2bd93-b748-40b0-8374-4be4852831c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6075ad911440ee89012c0ee4dde156'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:32.033824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ab2bd93-b748-40b0-8374-4be4852831c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6075ad911440ee89012c0ee4dde156'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:32.042919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1219/2453] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:31 2022] DELETE /identity/v3/users/7d6075ad911440ee89012c0ee4dde156 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:32.443642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:32.445173 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:32.462353 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10806f7f-3b6b-40e8-8e06-b0228e2a8417 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:32.521224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:32.537361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:32.538454 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:32.539592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:32.539954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] PATH_INFO: `/v3/projects/9eff3112495d4fd98391d9d392d75125` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:32.541271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eff3112495d4fd98391d9d392d75125)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:32.543320 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:32.606537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eff3112495d4fd98391d9d392d75125', 'request_id': 'req-0f7930b0-f703-4205-b8f7-05e6609bdb05'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:32.614895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eff3112495d4fd98391d9d392d75125', 'request_id': 'req-0f7930b0-f703-4205-b8f7-05e6609bdb05'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:32.624235 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eff3112495d4fd98391d9d392d75125', 'request_id': 'req-0f7930b0-f703-4205-b8f7-05e6609bdb05'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:32.641230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] The token cache is being invalidate because project 9eff3112495d4fd98391d9d392d75125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:32.641801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:32.642680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0f7930b0-f703-4205-b8f7-05e6609bdb05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:32.646118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1235/2454] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:32 2022] DELETE /identity/v3/projects/9eff3112495d4fd98391d9d392d75125 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:32.651323 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdcfec-5e41-456f-a74e-d2d97d7508f6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:32.651633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdcfec-5e41-456f-a74e-d2d97d7508f6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:32.651940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdcfec-5e41-456f-a74e-d2d97d7508f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:32.690807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b6bdcfec-5e41-456f-a74e-d2d97d7508f6 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:32.692912 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b6bdcfec-5e41-456f-a74e-d2d97d7508f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:32.792408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1220/2455] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:32.793945 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404f49-30a4-48f7-8e35-c743b51d4f50 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:32.794380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404f49-30a4-48f7-8e35-c743b51d4f50 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:32.794719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404f49-30a4-48f7-8e35-c743b51d4f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:32.820412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c9404f49-30a4-48f7-8e35-c743b51d4f50 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:32.822389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c9404f49-30a4-48f7-8e35-c743b51d4f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:32.833979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1236/2456] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:32.839757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdcfec-5e41-456f-a74e-d2d97d7508f6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:32.846381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:32.868392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdcfec-5e41-456f-a74e-d2d97d7508f6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:32.883180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce40ef3-5c11-46fb-bc2f-66f557bd47b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:32.884404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce40ef3-5c11-46fb-bc2f-66f557bd47b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:32.884700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce40ef3-5c11-46fb-bc2f-66f557bd47b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:32.884988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce40ef3-5c11-46fb-bc2f-66f557bd47b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:32.885757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce40ef3-5c11-46fb-bc2f-66f557bd47b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:06:32.886055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce40ef3-5c11-46fb-bc2f-66f557bd47b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:32.888116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce40ef3-5c11-46fb-bc2f-66f557bd47b5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:32.908367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1221/2457] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:06:32.909805 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9404f49-30a4-48f7-8e35-c743b51d4f50 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:32.929282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9404f49-30a4-48f7-8e35-c743b51d4f50 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:32.935901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd6ff2-172f-4c91-8ffe-63d2c6b001ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:32.936662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd6ff2-172f-4c91-8ffe-63d2c6b001ca admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:32.936994 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd6ff2-172f-4c91-8ffe-63d2c6b001ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:32.938037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd6ff2-172f-4c91-8ffe-63d2c6b001ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:32.940088 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd6ff2-172f-4c91-8ffe-63d2c6b001ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:32.942082 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd6ff2-172f-4c91-8ffe-63d2c6b001ca admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:32.978568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1237/2458] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:32 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:32.982442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce40ef3-5c11-46fb-bc2f-66f557bd47b5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:32.994587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce40ef3-5c11-46fb-bc2f-66f557bd47b5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:33.064191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacc55a-8a0e-46e1-ba76-a29ff8c2991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:33.064902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacc55a-8a0e-46e1-ba76-a29ff8c2991b admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.065281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacc55a-8a0e-46e1-ba76-a29ff8c2991b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.065625 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacc55a-8a0e-46e1-ba76-a29ff8c2991b admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.066414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacc55a-8a0e-46e1-ba76-a29ff8c2991b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:33.068682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacc55a-8a0e-46e1-ba76-a29ff8c2991b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:33.099461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1222/2459] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:32 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:33.100693 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd6ff2-172f-4c91-8ffe-63d2c6b001ca admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:33.111788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd6ff2-172f-4c91-8ffe-63d2c6b001ca admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:33.116739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c2ee-5a28-4465-b75f-2e185f65d504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:33.117363 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c2ee-5a28-4465-b75f-2e185f65d504 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.117650 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c2ee-5a28-4465-b75f-2e185f65d504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.117932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c2ee-5a28-4465-b75f-2e185f65d504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.118635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026c2ee-5a28-4465-b75f-2e185f65d504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:33.120266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026c2ee-5a28-4465-b75f-2e185f65d504 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:33.130775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1238/2460] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:33.135055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacc55a-8a0e-46e1-ba76-a29ff8c2991b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:33.146754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacc55a-8a0e-46e1-ba76-a29ff8c2991b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:33.151530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4f884-d3cc-45bd-b10a-db88b1706f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:33.152202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4f884-d3cc-45bd-b10a-db88b1706f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.152491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4f884-d3cc-45bd-b10a-db88b1706f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.152770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4f884-d3cc-45bd-b10a-db88b1706f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.153473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4f884-d3cc-45bd-b10a-db88b1706f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:33.155125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4f884-d3cc-45bd-b10a-db88b1706f91 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:33.168203 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1223/2461] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:33.172574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c2ee-5a28-4465-b75f-2e185f65d504 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:33.183658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c2ee-5a28-4465-b75f-2e185f65d504 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:33.189653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b0329-a230-48a9-8b13-9601ec5eb8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:33.190390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b0329-a230-48a9-8b13-9601ec5eb8f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.190916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b0329-a230-48a9-8b13-9601ec5eb8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.191275 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b0329-a230-48a9-8b13-9601ec5eb8f3 admin admin] PATH_INFO: `/v3/projects/a9d1ec09e1814a4f9cecc14d8baa91ab/users/49db3994ffcc4e8990904e8a7a19bf98/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.207866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b0329-a230-48a9-8b13-9601ec5eb8f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d1ec09e1814a4f9cecc14d8baa91ab, user_id=49db3994ffcc4e8990904e8a7a19bf98, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:33.210549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b0329-a230-48a9-8b13-9601ec5eb8f3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:33.224619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1239/2462] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:33 2022] PUT /identity/v3/projects/a9d1ec09e1814a4f9cecc14d8baa91ab/users/49db3994ffcc4e8990904e8a7a19bf98/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:33.235321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe8302-a30d-4a13-886c-65d56a03f092 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.235615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe8302-a30d-4a13-886c-65d56a03f092 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.235891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe8302-a30d-4a13-886c-65d56a03f092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.274588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b5fe8302-a30d-4a13-886c-65d56a03f092 None None] MFA Rules not processed for user `49db3994ffcc4e8990904e8a7a19bf98`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:33.276280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b5fe8302-a30d-4a13-886c-65d56a03f092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:33.382348 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1224/2463] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:33.383356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b0329-a230-48a9-8b13-9601ec5eb8f3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:33.462267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b0329-a230-48a9-8b13-9601ec5eb8f3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:33.467180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d2ec5-423d-4fdc-9813-ab21b32748e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:33.467804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d2ec5-423d-4fdc-9813-ab21b32748e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.468092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d2ec5-423d-4fdc-9813-ab21b32748e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.468392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d2ec5-423d-4fdc-9813-ab21b32748e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.469066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d2ec5-423d-4fdc-9813-ab21b32748e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:33.471142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d2ec5-423d-4fdc-9813-ab21b32748e4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:33.507582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1240/2464] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:33 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:06:33.508799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe8302-a30d-4a13-886c-65d56a03f092 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:33.520402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe8302-a30d-4a13-886c-65d56a03f092 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:33.586738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:33.595874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae2d54-1a11-45e2-a1a1-1eb1830a176a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:33.596509 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae2d54-1a11-45e2-a1a1-1eb1830a176a admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.596804 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae2d54-1a11-45e2-a1a1-1eb1830a176a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.597141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae2d54-1a11-45e2-a1a1-1eb1830a176a admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.598031 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae2d54-1a11-45e2-a1a1-1eb1830a176a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:33.599858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae2d54-1a11-45e2-a1a1-1eb1830a176a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:33.630740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1225/2465] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:33 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:33.634777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d2ec5-423d-4fdc-9813-ab21b32748e4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:33.643075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:33.649109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d2ec5-423d-4fdc-9813-ab21b32748e4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:33.655214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198d7b3-b955-43ff-8e6f-0389649898c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:33.655813 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198d7b3-b955-43ff-8e6f-0389649898c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.656150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198d7b3-b955-43ff-8e6f-0389649898c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.656429 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198d7b3-b955-43ff-8e6f-0389649898c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.657232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198d7b3-b955-43ff-8e6f-0389649898c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:33.659084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198d7b3-b955-43ff-8e6f-0389649898c3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:33.672140 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1241/2466] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:33.676757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae2d54-1a11-45e2-a1a1-1eb1830a176a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:33.687877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae2d54-1a11-45e2-a1a1-1eb1830a176a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:33.692848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0c6a1-8364-4a2f-bafe-eb03be02ea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:33.693615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0c6a1-8364-4a2f-bafe-eb03be02ea3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.694257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0c6a1-8364-4a2f-bafe-eb03be02ea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.694257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0c6a1-8364-4a2f-bafe-eb03be02ea3b admin admin] PATH_INFO: `/v3/projects/81996aab49bc4a04b1ed613b1ae6b1f8/users/7b79c4e4027c4df4b69cd2459aefb619/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.714965 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0c6a1-8364-4a2f-bafe-eb03be02ea3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=81996aab49bc4a04b1ed613b1ae6b1f8, user_id=7b79c4e4027c4df4b69cd2459aefb619, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:33.717797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0c6a1-8364-4a2f-bafe-eb03be02ea3b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:33.740778 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1226/2467] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:33 2022] PUT /identity/v3/projects/81996aab49bc4a04b1ed613b1ae6b1f8/users/7b79c4e4027c4df4b69cd2459aefb619/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:33.744997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198d7b3-b955-43ff-8e6f-0389649898c3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:33.758148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198d7b3-b955-43ff-8e6f-0389649898c3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:33.831705 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15917c1f-79cb-4066-9c21-355d50ec6896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:33.832561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917c1f-79cb-4066-9c21-355d50ec6896 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.832937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917c1f-79cb-4066-9c21-355d50ec6896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.833286 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917c1f-79cb-4066-9c21-355d50ec6896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.834074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15917c1f-79cb-4066-9c21-355d50ec6896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:33.836000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15917c1f-79cb-4066-9c21-355d50ec6896 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:33.851658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1242/2468] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:33.852542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0c6a1-8364-4a2f-bafe-eb03be02ea3b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:33.870468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0c6a1-8364-4a2f-bafe-eb03be02ea3b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:33.888243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f168c-379b-483a-94c7-c5ff6d37a03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:33.889131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f168c-379b-483a-94c7-c5ff6d37a03d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.889474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f168c-379b-483a-94c7-c5ff6d37a03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.889758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f168c-379b-483a-94c7-c5ff6d37a03d admin admin] PATH_INFO: `/v3/projects/81996aab49bc4a04b1ed613b1ae6b1f8/users/7b79c4e4027c4df4b69cd2459aefb619/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.892538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f168c-379b-483a-94c7-c5ff6d37a03d admin admin] RBAC: Authorizing `identity:create_grant(project_id=81996aab49bc4a04b1ed613b1ae6b1f8, user_id=7b79c4e4027c4df4b69cd2459aefb619, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:33.895751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f168c-379b-483a-94c7-c5ff6d37a03d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:33.913265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1227/2469] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:33 2022] PUT /identity/v3/projects/81996aab49bc4a04b1ed613b1ae6b1f8/users/7b79c4e4027c4df4b69cd2459aefb619/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:33.926256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc748f1-06a2-4623-a84d-bb1cf2217b12 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:33.926657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc748f1-06a2-4623-a84d-bb1cf2217b12 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:33.927029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc748f1-06a2-4623-a84d-bb1cf2217b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:33.977125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5fc748f1-06a2-4623-a84d-bb1cf2217b12 None None] MFA Rules not processed for user `7b79c4e4027c4df4b69cd2459aefb619`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:33.978873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5fc748f1-06a2-4623-a84d-bb1cf2217b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:34.103975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f168c-379b-483a-94c7-c5ff6d37a03d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:34.106014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ce4f168c-379b-483a-94c7-c5ff6d37a03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:34.125642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1243/2470] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:34.137810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc748f1-06a2-4623-a84d-bb1cf2217b12 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:34.139751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5fc748f1-06a2-4623-a84d-bb1cf2217b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:34.198777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f168c-379b-483a-94c7-c5ff6d37a03d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:34.207215 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056c0b6-6ed3-4c16-a49e-e82ed72ed63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:34.207859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056c0b6-6ed3-4c16-a49e-e82ed72ed63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:34.208156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056c0b6-6ed3-4c16-a49e-e82ed72ed63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:34.208402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056c0b6-6ed3-4c16-a49e-e82ed72ed63f admin admin] PATH_INFO: `/v3/users/c3308761e9d2495e95e127bbc7b9c5e2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:34.209610 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056c0b6-6ed3-4c16-a49e-e82ed72ed63f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3308761e9d2495e95e127bbc7b9c5e2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:34.211560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056c0b6-6ed3-4c16-a49e-e82ed72ed63f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:34.217993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc748f1-06a2-4623-a84d-bb1cf2217b12 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:34.221902 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd9427-38b6-4f23-8320-4269073d75ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:34.222553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd9427-38b6-4f23-8320-4269073d75ed service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:34.222878 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd9427-38b6-4f23-8320-4269073d75ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:34.223162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd9427-38b6-4f23-8320-4269073d75ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:34.255962 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:34.275167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7056c0b6-6ed3-4c16-a49e-e82ed72ed63f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3308761e9d2495e95e127bbc7b9c5e2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:34.276040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7056c0b6-6ed3-4c16-a49e-e82ed72ed63f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3308761e9d2495e95e127bbc7b9c5e2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:34.288859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7056c0b6-6ed3-4c16-a49e-e82ed72ed63f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3308761e9d2495e95e127bbc7b9c5e2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:34.294049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd9427-38b6-4f23-8320-4269073d75ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:34.295656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd9427-38b6-4f23-8320-4269073d75ed service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:34.306749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1244/2471] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:06:34.309180 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7056c0b6-6ed3-4c16-a49e-e82ed72ed63f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3308761e9d2495e95e127bbc7b9c5e2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:34.323889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7056c0b6-6ed3-4c16-a49e-e82ed72ed63f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3308761e9d2495e95e127bbc7b9c5e2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:34.330536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1228/2472] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:34 2022] DELETE /identity/v3/users/c3308761e9d2495e95e127bbc7b9c5e2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:34.514284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd9427-38b6-4f23-8320-4269073d75ed service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:34.528818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd9427-38b6-4f23-8320-4269073d75ed service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:34.607846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:34.608561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:34.608911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:34.609415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] PATH_INFO: `/v3/projects/18a927e9f5da42d7b34752c8b2db211a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:34.610806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=18a927e9f5da42d7b34752c8b2db211a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:34.612862 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:34.671290 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18a927e9f5da42d7b34752c8b2db211a', 'request_id': 'req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:34.677555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18a927e9f5da42d7b34752c8b2db211a', 'request_id': 'req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:34.683981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18a927e9f5da42d7b34752c8b2db211a', 'request_id': 'req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:34.697078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] The token cache is being invalidate because project 18a927e9f5da42d7b34752c8b2db211a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:34.697364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:34.698005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2ca0401b-41cc-46ef-bd6d-0d1b4777b69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:34.699656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1245/2473] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:34 2022] DELETE /identity/v3/projects/18a927e9f5da42d7b34752c8b2db211a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:34.704693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7016-7fcc-4519-9098-07fafe6a4831 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:34.705002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7016-7fcc-4519-9098-07fafe6a4831 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:34.705273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7016-7fcc-4519-9098-07fafe6a4831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:34.724160 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-62ee7016-7fcc-4519-9098-07fafe6a4831 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:34.725466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-62ee7016-7fcc-4519-9098-07fafe6a4831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:34.749519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:34.786382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1229/2474] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:34.794121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d745942-6571-4348-8437-cced5a9ffd19 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:34.794409 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d745942-6571-4348-8437-cced5a9ffd19 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:34.794689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d745942-6571-4348-8437-cced5a9ffd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:34.816179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0d745942-6571-4348-8437-cced5a9ffd19 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:34.817383 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0d745942-6571-4348-8437-cced5a9ffd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:34.825198 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1246/2475] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:34.829886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee7016-7fcc-4519-9098-07fafe6a4831 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:34.839614 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee7016-7fcc-4519-9098-07fafe6a4831 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:34.845214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849d9d0-b565-4dfb-b003-b661ec084170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:34.845819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849d9d0-b565-4dfb-b003-b661ec084170 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:34.846129 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849d9d0-b565-4dfb-b003-b661ec084170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:34.846403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849d9d0-b565-4dfb-b003-b661ec084170 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:34.847146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849d9d0-b565-4dfb-b003-b661ec084170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:06:34.847454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849d9d0-b565-4dfb-b003-b661ec084170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:34.849048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849d9d0-b565-4dfb-b003-b661ec084170 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:34.865185 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1230/2476] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:06:34.866374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d745942-6571-4348-8437-cced5a9ffd19 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:34.876603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d745942-6571-4348-8437-cced5a9ffd19 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:34.882371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d5b55-3739-4847-b929-3d324142546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:34.883006 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d5b55-3739-4847-b929-3d324142546f admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:34.883287 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d5b55-3739-4847-b929-3d324142546f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:34.883558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d5b55-3739-4847-b929-3d324142546f admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:34.884217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d5b55-3739-4847-b929-3d324142546f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:34.886101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d5b55-3739-4847-b929-3d324142546f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:34.917207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1247/2477] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:34 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:06:34.921409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849d9d0-b565-4dfb-b003-b661ec084170 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:34.931948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849d9d0-b565-4dfb-b003-b661ec084170 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:34.988092 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9cd498-470f-4767-8722-05f030469dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:34.988708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9cd498-470f-4767-8722-05f030469dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:34.988990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9cd498-470f-4767-8722-05f030469dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:34.989271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9cd498-470f-4767-8722-05f030469dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:34.989924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9cd498-470f-4767-8722-05f030469dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:34.991874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9cd498-470f-4767-8722-05f030469dc0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:35.013454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1231/2478] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:34 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:35.017840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d5b55-3739-4847-b929-3d324142546f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:35.028202 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d5b55-3739-4847-b929-3d324142546f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:35.033159 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce66e42-eeb6-477f-9775-1222c8b2e107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:35.033763 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce66e42-eeb6-477f-9775-1222c8b2e107 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:35.034046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce66e42-eeb6-477f-9775-1222c8b2e107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:35.034323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce66e42-eeb6-477f-9775-1222c8b2e107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:35.035090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce66e42-eeb6-477f-9775-1222c8b2e107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:35.036951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce66e42-eeb6-477f-9775-1222c8b2e107 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:35.047095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1248/2479] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:35.050958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9cd498-470f-4767-8722-05f030469dc0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:35.060828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9cd498-470f-4767-8722-05f030469dc0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:35.066014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6e12a-f826-46a3-9518-67d985507237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:35.066621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6e12a-f826-46a3-9518-67d985507237 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:35.066938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6e12a-f826-46a3-9518-67d985507237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:35.067215 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6e12a-f826-46a3-9518-67d985507237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:35.067904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6e12a-f826-46a3-9518-67d985507237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:35.069489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6e12a-f826-46a3-9518-67d985507237 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:35.080468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1232/2480] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:35.084527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce66e42-eeb6-477f-9775-1222c8b2e107 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:35.094477 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce66e42-eeb6-477f-9775-1222c8b2e107 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:35.099260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101f0e8-ed75-4327-82c6-83f169006901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:35.099926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101f0e8-ed75-4327-82c6-83f169006901 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:35.100211 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101f0e8-ed75-4327-82c6-83f169006901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:35.100491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101f0e8-ed75-4327-82c6-83f169006901 admin admin] PATH_INFO: `/v3/projects/be87f30a828c43c8834a92b5403473b9/users/d8f95968fe594dd4801e410738ca757a/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:35.115306 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101f0e8-ed75-4327-82c6-83f169006901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be87f30a828c43c8834a92b5403473b9, user_id=d8f95968fe594dd4801e410738ca757a, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:35.117944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101f0e8-ed75-4327-82c6-83f169006901 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:35.120916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:35.133811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1249/2481] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:35 2022] PUT /identity/v3/projects/be87f30a828c43c8834a92b5403473b9/users/d8f95968fe594dd4801e410738ca757a/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:06:35.144659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16061f4-1787-4a85-bc73-a07f29481af1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:35.144945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16061f4-1787-4a85-bc73-a07f29481af1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:35.145220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16061f4-1787-4a85-bc73-a07f29481af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:35.173698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e16061f4-1787-4a85-bc73-a07f29481af1 None None] MFA Rules not processed for user `d8f95968fe594dd4801e410738ca757a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:35.174945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e16061f4-1787-4a85-bc73-a07f29481af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:35.242061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1233/2482] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:35.254061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101f0e8-ed75-4327-82c6-83f169006901 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:35.256402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b101f0e8-ed75-4327-82c6-83f169006901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:35.314394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101f0e8-ed75-4327-82c6-83f169006901 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:35.318299 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87c4c8-b915-4785-8b07-7e3f5d558e4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:35.318988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87c4c8-b915-4785-8b07-7e3f5d558e4e service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:35.319276 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87c4c8-b915-4785-8b07-7e3f5d558e4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:35.319555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87c4c8-b915-4785-8b07-7e3f5d558e4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:35.331376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87c4c8-b915-4785-8b07-7e3f5d558e4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:35.332869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87c4c8-b915-4785-8b07-7e3f5d558e4e service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:35.343573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1250/2483] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:36.892028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16061f4-1787-4a85-bc73-a07f29481af1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:36.893643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e16061f4-1787-4a85-bc73-a07f29481af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:36.925089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:36.966188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16061f4-1787-4a85-bc73-a07f29481af1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:36.971723 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:36.972503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:36.972796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:36.973133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] PATH_INFO: `/v3/users/49db3994ffcc4e8990904e8a7a19bf98` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:36.974257 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49db3994ffcc4e8990904e8a7a19bf98)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:36.976233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:37.049231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49db3994ffcc4e8990904e8a7a19bf98'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.052155 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49db3994ffcc4e8990904e8a7a19bf98'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.060431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49db3994ffcc4e8990904e8a7a19bf98'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.075035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49db3994ffcc4e8990904e8a7a19bf98'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.082531 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49db3994ffcc4e8990904e8a7a19bf98'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.088590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1234/2484] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:36 2022] DELETE /identity/v3/users/49db3994ffcc4e8990904e8a7a19bf98 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:37.093069 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb774139-44c1-4bd5-b269-dcfbc1e02c88 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:37.094564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb774139-44c1-4bd5-b269-dcfbc1e02c88 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:37.094564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb774139-44c1-4bd5-b269-dcfbc1e02c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:37.113648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cb774139-44c1-4bd5-b269-dcfbc1e02c88 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:37.114933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cb774139-44c1-4bd5-b269-dcfbc1e02c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:37.175131 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1251/2485] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:37.184805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:37.186227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:37.244731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431429c5-a692-437c-ab37-0423deda0be3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:37.248459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1b015-7710-43f2-82c9-a4e36219054a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:37.249073 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1b015-7710-43f2-82c9-a4e36219054a service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:37.249368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1b015-7710-43f2-82c9-a4e36219054a service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:37.249747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1b015-7710-43f2-82c9-a4e36219054a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:37.258479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1b015-7710-43f2-82c9-a4e36219054a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:37.259943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1b015-7710-43f2-82c9-a4e36219054a service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:37.269265 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1235/2486] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:37.541331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb774139-44c1-4bd5-b269-dcfbc1e02c88 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:37.542846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cb774139-44c1-4bd5-b269-dcfbc1e02c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:37.555402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb774139-44c1-4bd5-b269-dcfbc1e02c88 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:37.560270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:37.560920 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:37.561204 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:37.561482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] PATH_INFO: `/v3/projects/a9d1ec09e1814a4f9cecc14d8baa91ab` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:37.563075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9d1ec09e1814a4f9cecc14d8baa91ab)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:37.565133 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:37.611806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9d1ec09e1814a4f9cecc14d8baa91ab', 'request_id': 'req-0e517e5f-dd5e-47ba-9cab-ae8e239925df'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.617272 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9d1ec09e1814a4f9cecc14d8baa91ab', 'request_id': 'req-0e517e5f-dd5e-47ba-9cab-ae8e239925df'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.623323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9d1ec09e1814a4f9cecc14d8baa91ab', 'request_id': 'req-0e517e5f-dd5e-47ba-9cab-ae8e239925df'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.636322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] The token cache is being invalidate because project a9d1ec09e1814a4f9cecc14d8baa91ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:37.636606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.637223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.638723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1252/2487] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:37 2022] DELETE /identity/v3/projects/a9d1ec09e1814a4f9cecc14d8baa91ab => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:37.643394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1b015-7710-43f2-82c9-a4e36219054a service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:37.652926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1b015-7710-43f2-82c9-a4e36219054a service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:37.695926 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:37.709993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cd395-7ee3-4ce4-a31d-f73292731aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:37.710623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cd395-7ee3-4ce4-a31d-f73292731aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:37.710937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cd395-7ee3-4ce4-a31d-f73292731aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:37.711217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cd395-7ee3-4ce4-a31d-f73292731aac admin admin] PATH_INFO: `/v3/users/7b79c4e4027c4df4b69cd2459aefb619` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:37.712419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8cd395-7ee3-4ce4-a31d-f73292731aac admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b79c4e4027c4df4b69cd2459aefb619)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:37.714325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8cd395-7ee3-4ce4-a31d-f73292731aac admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:37.763429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ff8cd395-7ee3-4ce4-a31d-f73292731aac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b79c4e4027c4df4b69cd2459aefb619'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.764289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ff8cd395-7ee3-4ce4-a31d-f73292731aac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b79c4e4027c4df4b69cd2459aefb619'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.772111 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ff8cd395-7ee3-4ce4-a31d-f73292731aac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b79c4e4027c4df4b69cd2459aefb619'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.786859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ff8cd395-7ee3-4ce4-a31d-f73292731aac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b79c4e4027c4df4b69cd2459aefb619'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.793677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ff8cd395-7ee3-4ce4-a31d-f73292731aac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b79c4e4027c4df4b69cd2459aefb619'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:37.799629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1236/2488] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:37 2022] DELETE /identity/v3/users/7b79c4e4027c4df4b69cd2459aefb619 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:38.124516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:38.126731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:38.140606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e517e5f-dd5e-47ba-9cab-ae8e239925df admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:38.168360 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:38.210513 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:38.211196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:38.211476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:38.211754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] PATH_INFO: `/v3/projects/81996aab49bc4a04b1ed613b1ae6b1f8` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:38.212941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] RBAC: Authorizing `identity:delete_project(project_id=81996aab49bc4a04b1ed613b1ae6b1f8)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:38.214882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:38.265466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81996aab49bc4a04b1ed613b1ae6b1f8', 'request_id': 'req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:38.271607 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81996aab49bc4a04b1ed613b1ae6b1f8', 'request_id': 'req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:38.277970 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81996aab49bc4a04b1ed613b1ae6b1f8', 'request_id': 'req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:38.291817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] The token cache is being invalidate because project 81996aab49bc4a04b1ed613b1ae6b1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:38.292085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:38.292786 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-988d16b3-ab3e-4ca7-8f70-3627a6aec51a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:38.294681 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1253/2489] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:38 2022] DELETE /identity/v3/projects/81996aab49bc4a04b1ed613b1ae6b1f8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:38.300969 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53d995-c118-434e-b649-2cac53108307 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:38.301424 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53d995-c118-434e-b649-2cac53108307 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:38.301621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53d995-c118-434e-b649-2cac53108307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:38.323522 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3f53d995-c118-434e-b649-2cac53108307 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:38.324754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3f53d995-c118-434e-b649-2cac53108307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:38.388059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1237/2490] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:38.399583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445eae89-8ebc-478a-b322-ea8d0f72a17c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:38.399869 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445eae89-8ebc-478a-b322-ea8d0f72a17c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:38.400153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445eae89-8ebc-478a-b322-ea8d0f72a17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:38.420569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-445eae89-8ebc-478a-b322-ea8d0f72a17c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:38.422487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-445eae89-8ebc-478a-b322-ea8d0f72a17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:38.431465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1254/2491] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:38.435703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53d995-c118-434e-b649-2cac53108307 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:38.448150 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53d995-c118-434e-b649-2cac53108307 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:38.459042 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c7f37-9fa0-4d00-af64-8e8df9fa0abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:38.459776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c7f37-9fa0-4d00-af64-8e8df9fa0abd admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:38.460359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c7f37-9fa0-4d00-af64-8e8df9fa0abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:38.460824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c7f37-9fa0-4d00-af64-8e8df9fa0abd admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:38.461677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c7f37-9fa0-4d00-af64-8e8df9fa0abd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:06:38.461988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c7f37-9fa0-4d00-af64-8e8df9fa0abd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:38.463752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c7f37-9fa0-4d00-af64-8e8df9fa0abd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:38.477600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1238/2492] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:06:38.481894 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445eae89-8ebc-478a-b322-ea8d0f72a17c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:38.493063 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445eae89-8ebc-478a-b322-ea8d0f72a17c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:38.498078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fa540-8d56-4dc7-9bc1-f2baaa852d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:38.498696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fa540-8d56-4dc7-9bc1-f2baaa852d89 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:38.499016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fa540-8d56-4dc7-9bc1-f2baaa852d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:38.499312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fa540-8d56-4dc7-9bc1-f2baaa852d89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:38.500788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fa540-8d56-4dc7-9bc1-f2baaa852d89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:38.502675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fa540-8d56-4dc7-9bc1-f2baaa852d89 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:38.529210 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1255/2493] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:38 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:06:38.534415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c7f37-9fa0-4d00-af64-8e8df9fa0abd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:38.545606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c7f37-9fa0-4d00-af64-8e8df9fa0abd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:38.595701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:38.606076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d72ee2-19fa-457c-a140-a573bf3eeb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:38.606703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d72ee2-19fa-457c-a140-a573bf3eeb07 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:38.607128 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d72ee2-19fa-457c-a140-a573bf3eeb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:38.607449 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d72ee2-19fa-457c-a140-a573bf3eeb07 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:38.608291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d72ee2-19fa-457c-a140-a573bf3eeb07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:38.610819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d72ee2-19fa-457c-a140-a573bf3eeb07 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:38.630947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1239/2494] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:38 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:38.635049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fa540-8d56-4dc7-9bc1-f2baaa852d89 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:38.645222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fa540-8d56-4dc7-9bc1-f2baaa852d89 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:38.649797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a5dcd-6bc4-4b70-b134-9b23a9768133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:38.650394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a5dcd-6bc4-4b70-b134-9b23a9768133 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:38.650673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a5dcd-6bc4-4b70-b134-9b23a9768133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:38.650979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a5dcd-6bc4-4b70-b134-9b23a9768133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:38.651668 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a5dcd-6bc4-4b70-b134-9b23a9768133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:38.653240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a5dcd-6bc4-4b70-b134-9b23a9768133 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:38.663977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1256/2495] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:38.668211 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d72ee2-19fa-457c-a140-a573bf3eeb07 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:38.679340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d72ee2-19fa-457c-a140-a573bf3eeb07 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:38.684033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d440c-a1fe-4f35-bcc0-8a47997186aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:38.684639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d440c-a1fe-4f35-bcc0-8a47997186aa admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:38.684923 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d440c-a1fe-4f35-bcc0-8a47997186aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:38.685204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d440c-a1fe-4f35-bcc0-8a47997186aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:38.685893 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d440c-a1fe-4f35-bcc0-8a47997186aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:38.687740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d440c-a1fe-4f35-bcc0-8a47997186aa admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:38.698344 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1240/2496] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:38.702469 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a5dcd-6bc4-4b70-b134-9b23a9768133 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:38.712776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a5dcd-6bc4-4b70-b134-9b23a9768133 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:38.717329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9e687-00cc-4b66-a0b1-e147a8cc90a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:38.717975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e687-00cc-4b66-a0b1-e147a8cc90a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:38.719282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e687-00cc-4b66-a0b1-e147a8cc90a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:38.719282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e687-00cc-4b66-a0b1-e147a8cc90a1 admin admin] PATH_INFO: `/v3/projects/1014657d65f143569266e26cf8ab4af3/users/29a7222d467b4d46a528fda84adc5f17/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:38.735226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9e687-00cc-4b66-a0b1-e147a8cc90a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1014657d65f143569266e26cf8ab4af3, user_id=29a7222d467b4d46a528fda84adc5f17, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:38.738096 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9e687-00cc-4b66-a0b1-e147a8cc90a1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:38.751490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1257/2497] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:38 2022] PUT /identity/v3/projects/1014657d65f143569266e26cf8ab4af3/users/29a7222d467b4d46a528fda84adc5f17/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:38.761648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fc34c-1978-475d-9589-54b60c025073 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:38.761955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fc34c-1978-475d-9589-54b60c025073 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:38.762231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fc34c-1978-475d-9589-54b60c025073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:38.790739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-921fc34c-1978-475d-9589-54b60c025073 None None] MFA Rules not processed for user `29a7222d467b4d46a528fda84adc5f17`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:38.792015 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-921fc34c-1978-475d-9589-54b60c025073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:38.855125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1241/2498] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:38.866810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9e687-00cc-4b66-a0b1-e147a8cc90a1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:38.868558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5cb9e687-00cc-4b66-a0b1-e147a8cc90a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:38.922845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:38.926804 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9e687-00cc-4b66-a0b1-e147a8cc90a1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:38.931204 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebae39-57fc-4e27-9bdb-341e422a0c4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:38.931960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebae39-57fc-4e27-9bdb-341e422a0c4b service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:38.932297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebae39-57fc-4e27-9bdb-341e422a0c4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:38.932780 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebae39-57fc-4e27-9bdb-341e422a0c4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:38.942479 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebae39-57fc-4e27-9bdb-341e422a0c4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:38.944472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebae39-57fc-4e27-9bdb-341e422a0c4b service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:38.953328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1258/2499] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:48.852299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:48.948988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:49.069791 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fc34c-1978-475d-9589-54b60c025073 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:49.071381 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-921fc34c-1978-475d-9589-54b60c025073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:49.137518 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fc34c-1978-475d-9589-54b60c025073 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:49.142544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:49.143217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:49.143500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:49.143773 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] PATH_INFO: `/v3/users/d8f95968fe594dd4801e410738ca757a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:49.144940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8f95968fe594dd4801e410738ca757a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:49.146902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:49.193348 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8f95968fe594dd4801e410738ca757a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:49.194005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8f95968fe594dd4801e410738ca757a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:49.200918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8f95968fe594dd4801e410738ca757a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:49.213336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8f95968fe594dd4801e410738ca757a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:49.220284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8f95968fe594dd4801e410738ca757a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:49.226070 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1242/2500] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:06:49 2022] DELETE /identity/v3/users/d8f95968fe594dd4801e410738ca757a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:49.230508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530b1c4-89fb-4f26-bd0d-86feeedd37b8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:49.230817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530b1c4-89fb-4f26-bd0d-86feeedd37b8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:49.231107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530b1c4-89fb-4f26-bd0d-86feeedd37b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:49.249817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8530b1c4-89fb-4f26-bd0d-86feeedd37b8 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:49.251595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8530b1c4-89fb-4f26-bd0d-86feeedd37b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:49.309556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1259/2501] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:49.319434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:49.320830 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:49.377254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088f9b5-82e6-4eee-82e5-e5375b56205b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:49.380868 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a179e-dc8d-48cf-9ffc-f107f2b9f98c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:49.381473 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a179e-dc8d-48cf-9ffc-f107f2b9f98c service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:49.381752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a179e-dc8d-48cf-9ffc-f107f2b9f98c service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:49.382028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a179e-dc8d-48cf-9ffc-f107f2b9f98c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:49.391027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a179e-dc8d-48cf-9ffc-f107f2b9f98c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:49.392940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a179e-dc8d-48cf-9ffc-f107f2b9f98c service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:49.402191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1243/2502] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:06:49.743617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530b1c4-89fb-4f26-bd0d-86feeedd37b8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:49.745131 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8530b1c4-89fb-4f26-bd0d-86feeedd37b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:49.768381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530b1c4-89fb-4f26-bd0d-86feeedd37b8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:49.774332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:49.774962 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:49.775252 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:49.775533 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] PATH_INFO: `/v3/projects/be87f30a828c43c8834a92b5403473b9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:49.776753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=be87f30a828c43c8834a92b5403473b9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:49.779118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:49.827871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be87f30a828c43c8834a92b5403473b9', 'request_id': 'req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:49.833561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be87f30a828c43c8834a92b5403473b9', 'request_id': 'req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:49.839706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be87f30a828c43c8834a92b5403473b9', 'request_id': 'req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:49.852916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] The token cache is being invalidate because project be87f30a828c43c8834a92b5403473b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:06:49.853236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:49.853884 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1a5b1b03-240f-44b2-b7bf-b7697f5132ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:06:49.855445 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1260/2503] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:06:49 2022] DELETE /identity/v3/projects/be87f30a828c43c8834a92b5403473b9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:49.860293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433930e9-cb3e-46e9-87bc-08ccc43cb8be None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:49.861478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433930e9-cb3e-46e9-87bc-08ccc43cb8be None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:49.861478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433930e9-cb3e-46e9-87bc-08ccc43cb8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:49.880549 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-433930e9-cb3e-46e9-87bc-08ccc43cb8be None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:49.881730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-433930e9-cb3e-46e9-87bc-08ccc43cb8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:49.898729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:49.943847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1244/2504] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:49.951692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563891be-68c0-4beb-98e2-2de894488d33 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:49.952170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563891be-68c0-4beb-98e2-2de894488d33 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:49.952280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563891be-68c0-4beb-98e2-2de894488d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:49.972087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-563891be-68c0-4beb-98e2-2de894488d33 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:49.973293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-563891be-68c0-4beb-98e2-2de894488d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:49.981423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1261/2505] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:49.986734 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433930e9-cb3e-46e9-87bc-08ccc43cb8be None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:49.996341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433930e9-cb3e-46e9-87bc-08ccc43cb8be None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.000875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54207311-4422-4853-887f-963566275f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.002027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54207311-4422-4853-887f-963566275f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.002310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54207311-4422-4853-887f-963566275f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.002586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54207311-4422-4853-887f-963566275f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.003321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54207311-4422-4853-887f-963566275f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:06:50.003630 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54207311-4422-4853-887f-963566275f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.005204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54207311-4422-4853-887f-963566275f18 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.016501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1245/2506] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:06:50.020656 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563891be-68c0-4beb-98e2-2de894488d33 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.029940 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563891be-68c0-4beb-98e2-2de894488d33 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.035209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647b78-eb62-44a8-9188-2cc0ec7ff539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.035816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647b78-eb62-44a8-9188-2cc0ec7ff539 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.036357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647b78-eb62-44a8-9188-2cc0ec7ff539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.036637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647b78-eb62-44a8-9188-2cc0ec7ff539 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.037312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12647b78-eb62-44a8-9188-2cc0ec7ff539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.039254 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12647b78-eb62-44a8-9188-2cc0ec7ff539 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.066856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1262/2507] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:50 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:06:50.070943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54207311-4422-4853-887f-963566275f18 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.081933 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54207311-4422-4853-887f-963566275f18 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.137852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5babfd-7b70-40f4-bff6-5b2efcb2fa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.138468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5babfd-7b70-40f4-bff6-5b2efcb2fa6c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.138755 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5babfd-7b70-40f4-bff6-5b2efcb2fa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.139068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5babfd-7b70-40f4-bff6-5b2efcb2fa6c admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.139737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5babfd-7b70-40f4-bff6-5b2efcb2fa6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.142151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5babfd-7b70-40f4-bff6-5b2efcb2fa6c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.164027 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1246/2508] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:50 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:50.168025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647b78-eb62-44a8-9188-2cc0ec7ff539 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.178348 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647b78-eb62-44a8-9188-2cc0ec7ff539 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.183328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448bbbc-1a6f-4899-82e5-d545e25ce252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.183955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448bbbc-1a6f-4899-82e5-d545e25ce252 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.184228 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448bbbc-1a6f-4899-82e5-d545e25ce252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.184507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448bbbc-1a6f-4899-82e5-d545e25ce252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.185212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448bbbc-1a6f-4899-82e5-d545e25ce252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.186823 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448bbbc-1a6f-4899-82e5-d545e25ce252 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.196676 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1263/2509] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:50.200563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5babfd-7b70-40f4-bff6-5b2efcb2fa6c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.211401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5babfd-7b70-40f4-bff6-5b2efcb2fa6c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.216040 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e6f36-7614-4cb2-b41b-c9c033d58b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.216639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e6f36-7614-4cb2-b41b-c9c033d58b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.216922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e6f36-7614-4cb2-b41b-c9c033d58b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.217199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e6f36-7614-4cb2-b41b-c9c033d58b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.217919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e6f36-7614-4cb2-b41b-c9c033d58b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.219541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e6f36-7614-4cb2-b41b-c9c033d58b4e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.230667 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1247/2510] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:50.234664 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448bbbc-1a6f-4899-82e5-d545e25ce252 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.245013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448bbbc-1a6f-4899-82e5-d545e25ce252 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.249637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876696e-2b69-4e80-82b7-132a915db8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.250285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876696e-2b69-4e80-82b7-132a915db8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.250568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876696e-2b69-4e80-82b7-132a915db8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.250874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876696e-2b69-4e80-82b7-132a915db8ba admin admin] PATH_INFO: `/v3/projects/b3c2f5b93f854f32bb6a5c8377539e3f/users/90b98366f0be45bfbad94a5f1d0fc2a0/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.265493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876696e-2b69-4e80-82b7-132a915db8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c2f5b93f854f32bb6a5c8377539e3f, user_id=90b98366f0be45bfbad94a5f1d0fc2a0, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.269179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876696e-2b69-4e80-82b7-132a915db8ba admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.283592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1264/2511] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:50 2022] PUT /identity/v3/projects/b3c2f5b93f854f32bb6a5c8377539e3f/users/90b98366f0be45bfbad94a5f1d0fc2a0/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:50.294106 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b1a32-cd84-4893-86fc-b2a9c31da161 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.294394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b1a32-cd84-4893-86fc-b2a9c31da161 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.294662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b1a32-cd84-4893-86fc-b2a9c31da161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.321433 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5e1b1a32-cd84-4893-86fc-b2a9c31da161 None None] MFA Rules not processed for user `90b98366f0be45bfbad94a5f1d0fc2a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:50.322658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5e1b1a32-cd84-4893-86fc-b2a9c31da161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:50.386034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1248/2512] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:50.390682 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876696e-2b69-4e80-82b7-132a915db8ba admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.413918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:50.458258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876696e-2b69-4e80-82b7-132a915db8ba admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.462848 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914c5e8-4a62-41da-a574-5ec1e2ee3ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.463507 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914c5e8-4a62-41da-a574-5ec1e2ee3ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.463791 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914c5e8-4a62-41da-a574-5ec1e2ee3ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.464064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914c5e8-4a62-41da-a574-5ec1e2ee3ba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.464733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914c5e8-4a62-41da-a574-5ec1e2ee3ba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.466628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914c5e8-4a62-41da-a574-5ec1e2ee3ba1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.493032 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1265/2513] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:06:50 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:50.497145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b1a32-cd84-4893-86fc-b2a9c31da161 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.508318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b1a32-cd84-4893-86fc-b2a9c31da161 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.520605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:50.569395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ffa5c-7d6e-4d32-805a-31428c002848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.570028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ffa5c-7d6e-4d32-805a-31428c002848 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.570315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ffa5c-7d6e-4d32-805a-31428c002848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.570593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ffa5c-7d6e-4d32-805a-31428c002848 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.571307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ffa5c-7d6e-4d32-805a-31428c002848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.573291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ffa5c-7d6e-4d32-805a-31428c002848 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.594144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1249/2514] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:06:50 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:06:50.598670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914c5e8-4a62-41da-a574-5ec1e2ee3ba1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.608902 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914c5e8-4a62-41da-a574-5ec1e2ee3ba1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.614161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0dd16-0f20-4d41-b7e5-b8f63105efbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.614761 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0dd16-0f20-4d41-b7e5-b8f63105efbb admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.615071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0dd16-0f20-4d41-b7e5-b8f63105efbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.615345 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0dd16-0f20-4d41-b7e5-b8f63105efbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.616035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0dd16-0f20-4d41-b7e5-b8f63105efbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.617615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0dd16-0f20-4d41-b7e5-b8f63105efbb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.628487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1266/2515] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:50.632429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ffa5c-7d6e-4d32-805a-31428c002848 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.642552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ffa5c-7d6e-4d32-805a-31428c002848 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.647362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ece95c-e2ad-4784-91ba-5c3430def6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.648011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ece95c-e2ad-4784-91ba-5c3430def6cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.648290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ece95c-e2ad-4784-91ba-5c3430def6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.648563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ece95c-e2ad-4784-91ba-5c3430def6cb admin admin] PATH_INFO: `/v3/projects/0b5790108be94ccbad9c9674d04b4123/users/a0133f7b523f4db18748158bb74e13c1/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.663529 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ece95c-e2ad-4784-91ba-5c3430def6cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5790108be94ccbad9c9674d04b4123, user_id=a0133f7b523f4db18748158bb74e13c1, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.666134 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ece95c-e2ad-4784-91ba-5c3430def6cb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.679217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1250/2516] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:50 2022] PUT /identity/v3/projects/0b5790108be94ccbad9c9674d04b4123/users/a0133f7b523f4db18748158bb74e13c1/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:50.683434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0dd16-0f20-4d41-b7e5-b8f63105efbb admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.693369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0dd16-0f20-4d41-b7e5-b8f63105efbb admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.749095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87dd9dc-7d85-4008-b215-f95ca8792f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.749640 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dd9dc-7d85-4008-b215-f95ca8792f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.749919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dd9dc-7d85-4008-b215-f95ca8792f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.750193 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dd9dc-7d85-4008-b215-f95ca8792f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.751011 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87dd9dc-7d85-4008-b215-f95ca8792f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.752584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87dd9dc-7d85-4008-b215-f95ca8792f87 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.761991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1267/2517] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:06:50.766124 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ece95c-e2ad-4784-91ba-5c3430def6cb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.777449 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ece95c-e2ad-4784-91ba-5c3430def6cb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.783126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb44f9e-dc3a-46b0-aef7-ef4cb598026f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.784049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb44f9e-dc3a-46b0-aef7-ef4cb598026f admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.784331 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb44f9e-dc3a-46b0-aef7-ef4cb598026f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.784747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb44f9e-dc3a-46b0-aef7-ef4cb598026f admin admin] PATH_INFO: `/v3/projects/0b5790108be94ccbad9c9674d04b4123/users/a0133f7b523f4db18748158bb74e13c1/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.787259 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb44f9e-dc3a-46b0-aef7-ef4cb598026f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5790108be94ccbad9c9674d04b4123, user_id=a0133f7b523f4db18748158bb74e13c1, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:50.790078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb44f9e-dc3a-46b0-aef7-ef4cb598026f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:50.801533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1251/2518] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:06:50 2022] PUT /identity/v3/projects/0b5790108be94ccbad9c9674d04b4123/users/a0133f7b523f4db18748158bb74e13c1/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:06:50.812991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39790a91-f577-42a6-94bd-9e72ab83e139 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.813280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39790a91-f577-42a6-94bd-9e72ab83e139 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.813603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39790a91-f577-42a6-94bd-9e72ab83e139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:50.841720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-39790a91-f577-42a6-94bd-9e72ab83e139 None None] MFA Rules not processed for user `a0133f7b523f4db18748158bb74e13c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:50.842955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-39790a91-f577-42a6-94bd-9e72ab83e139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:50.915765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1268/2519] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:06:50 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:50.927701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb44f9e-dc3a-46b0-aef7-ef4cb598026f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:50.943785 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1cb44f9e-dc3a-46b0-aef7-ef4cb598026f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:50.990221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb44f9e-dc3a-46b0-aef7-ef4cb598026f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:50.993825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604821b-97f1-4202-9f5f-c4237a9e956d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:50.994436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604821b-97f1-4202-9f5f-c4237a9e956d service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:50.994719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604821b-97f1-4202-9f5f-c4237a9e956d service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:50.995029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604821b-97f1-4202-9f5f-c4237a9e956d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:51.004033 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604821b-97f1-4202-9f5f-c4237a9e956d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:51.005400 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604821b-97f1-4202-9f5f-c4237a9e956d service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:51.014752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1252/2520] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:50 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:51.089576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39790a91-f577-42a6-94bd-9e72ab83e139 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:51.099865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39790a91-f577-42a6-94bd-9e72ab83e139 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:51.103554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca7d98-282d-4514-9300-c6c5586d8819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:51.104167 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca7d98-282d-4514-9300-c6c5586d8819 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:51.104448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca7d98-282d-4514-9300-c6c5586d8819 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:51.104721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca7d98-282d-4514-9300-c6c5586d8819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:51.117097 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:06:51.162840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca7d98-282d-4514-9300-c6c5586d8819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:51.164308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca7d98-282d-4514-9300-c6c5586d8819 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:51.173843 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1269/2521] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:06:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:06:53.537758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27d8e7-aa05-44f2-8b59-617baa532602 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:53.538231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27d8e7-aa05-44f2-8b59-617baa532602 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:53.538670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27d8e7-aa05-44f2-8b59-617baa532602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:53.562147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fa27d8e7-aa05-44f2-8b59-617baa532602 None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:06:53.563450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fa27d8e7-aa05-44f2-8b59-617baa532602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:53.623220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1253/2522] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 04:06:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:06:53.674094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca7d98-282d-4514-9300-c6c5586d8819 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:06:53.675954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-25ca7d98-282d-4514-9300-c6c5586d8819 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:06:53.756988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca7d98-282d-4514-9300-c6c5586d8819 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:06:53.762722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d86617-bf87-4b21-b0b9-32e5cbc393d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:06:53.763386 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d86617-bf87-4b21-b0b9-32e5cbc393d6 service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:06:53.763671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d86617-bf87-4b21-b0b9-32e5cbc393d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:06:53.763952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d86617-bf87-4b21-b0b9-32e5cbc393d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:06:53.772749 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d86617-bf87-4b21-b0b9-32e5cbc393d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:06:53.774534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d86617-bf87-4b21-b0b9-32e5cbc393d6 service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:06:53.783832 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1270/2523] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:06:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 04:07:00.602404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27d8e7-aa05-44f2-8b59-617baa532602 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:00.603891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fa27d8e7-aa05-44f2-8b59-617baa532602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:00.652430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:00.742555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27d8e7-aa05-44f2-8b59-617baa532602 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:00.747498 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:00.748255 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:00.748581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:00.748865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] PATH_INFO: `/v3/users/29a7222d467b4d46a528fda84adc5f17` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:00.750148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29a7222d467b4d46a528fda84adc5f17)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:00.752921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:00.811459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29a7222d467b4d46a528fda84adc5f17'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:00.812561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29a7222d467b4d46a528fda84adc5f17'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:00.821075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29a7222d467b4d46a528fda84adc5f17'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:00.837991 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29a7222d467b4d46a528fda84adc5f17'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:00.846846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29a7222d467b4d46a528fda84adc5f17'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:00.858573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1254/2524] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:00 2022] DELETE /identity/v3/users/29a7222d467b4d46a528fda84adc5f17 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:00.860877 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd572339-c4aa-4c1b-84a5-00395faeb44b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:00.861372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd572339-c4aa-4c1b-84a5-00395faeb44b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:00.861372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd572339-c4aa-4c1b-84a5-00395faeb44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:00.882999 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fd572339-c4aa-4c1b-84a5-00395faeb44b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:00.884282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fd572339-c4aa-4c1b-84a5-00395faeb44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:00.982357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1271/2525] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:07:00.990585 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:00.992386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:01.060413 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bba6d-e9f5-4f0b-8fe5-9463b84bf1a4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:01.065209 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87fad9-8b77-45d7-994c-094579314dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:01.065869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87fad9-8b77-45d7-994c-094579314dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:01.066196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87fad9-8b77-45d7-994c-094579314dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:01.066468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87fad9-8b77-45d7-994c-094579314dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:01.078424 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87fad9-8b77-45d7-994c-094579314dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:01.080327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87fad9-8b77-45d7-994c-094579314dc8 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:01.091652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1255/2526] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:01.427611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd572339-c4aa-4c1b-84a5-00395faeb44b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:01.429533 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fd572339-c4aa-4c1b-84a5-00395faeb44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:01.464029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd572339-c4aa-4c1b-84a5-00395faeb44b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:01.475870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:01.477029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:01.477561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:01.477906 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] PATH_INFO: `/v3/projects/1014657d65f143569266e26cf8ab4af3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:01.479378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1014657d65f143569266e26cf8ab4af3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:01.481602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:01.541117 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:01.553494 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1014657d65f143569266e26cf8ab4af3', 'request_id': 'req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:01.563892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1014657d65f143569266e26cf8ab4af3', 'request_id': 'req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:01.572109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1014657d65f143569266e26cf8ab4af3', 'request_id': 'req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:01.586326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] The token cache is being invalidate because project 1014657d65f143569266e26cf8ab4af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:01.587154 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:01.588059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8b4ce75d-eb07-4e71-a8a5-feaad6a47336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:01.590865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1272/2527] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:01 2022] DELETE /identity/v3/projects/1014657d65f143569266e26cf8ab4af3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:01.594886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884d90f-6469-4429-87a2-259cc96b9504 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:01.595178 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884d90f-6469-4429-87a2-259cc96b9504 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:01.595685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884d90f-6469-4429-87a2-259cc96b9504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:01.620039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f884d90f-6469-4429-87a2-259cc96b9504 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:01.621515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f884d90f-6469-4429-87a2-259cc96b9504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:01.664953 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:01.707132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1256/2528] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:01.709393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e975c-d3aa-430f-992c-4bf45618273f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:01.710130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e975c-d3aa-430f-992c-4bf45618273f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:01.710904 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e975c-d3aa-430f-992c-4bf45618273f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:01.740134 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-be4e975c-d3aa-430f-992c-4bf45618273f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:01.741484 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-be4e975c-d3aa-430f-992c-4bf45618273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:01.751020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1273/2529] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:01.756511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884d90f-6469-4429-87a2-259cc96b9504 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:01.773687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884d90f-6469-4429-87a2-259cc96b9504 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:01.781281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbcdf2d-b7a5-4363-b1a4-d618d61ead40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:01.782719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcdf2d-b7a5-4363-b1a4-d618d61ead40 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:01.783253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcdf2d-b7a5-4363-b1a4-d618d61ead40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:01.783963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcdf2d-b7a5-4363-b1a4-d618d61ead40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:01.785764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcdf2d-b7a5-4363-b1a4-d618d61ead40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:07:01.786563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcdf2d-b7a5-4363-b1a4-d618d61ead40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:01.788528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcdf2d-b7a5-4363-b1a4-d618d61ead40 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:01.823532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1257/2530] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:07:01.827287 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e975c-d3aa-430f-992c-4bf45618273f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:01.841844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e975c-d3aa-430f-992c-4bf45618273f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:01.854307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd1301-2c00-4ead-962e-a0e5249954f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:01.855525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd1301-2c00-4ead-962e-a0e5249954f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:01.856253 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd1301-2c00-4ead-962e-a0e5249954f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:01.857963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd1301-2c00-4ead-962e-a0e5249954f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:01.859586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd1301-2c00-4ead-962e-a0e5249954f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:01.865853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd1301-2c00-4ead-962e-a0e5249954f3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:01.921090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1274/2531] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:01 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:01.922508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbcdf2d-b7a5-4363-b1a4-d618d61ead40 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:01.936974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbcdf2d-b7a5-4363-b1a4-d618d61ead40 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:02.008230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd1301-2c00-4ead-962e-a0e5249954f3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:02.010060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e0dd1301-2c00-4ead-962e-a0e5249954f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:02.013489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3cd1f-f2e5-4a06-a5ee-d5694031ca30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:02.014120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3cd1f-f2e5-4a06-a5ee-d5694031ca30 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:02.014415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3cd1f-f2e5-4a06-a5ee-d5694031ca30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:02.014693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3cd1f-f2e5-4a06-a5ee-d5694031ca30 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:02.015406 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3cd1f-f2e5-4a06-a5ee-d5694031ca30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:02.017458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3cd1f-f2e5-4a06-a5ee-d5694031ca30 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:02.029698 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd1301-2c00-4ead-962e-a0e5249954f3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:02.037544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d420aa2-ca96-4788-8b86-9a311afb3617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:02.038243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d420aa2-ca96-4788-8b86-9a311afb3617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:02.038529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d420aa2-ca96-4788-8b86-9a311afb3617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:02.039027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d420aa2-ca96-4788-8b86-9a311afb3617 admin admin] PATH_INFO: `/v3/users/90b98366f0be45bfbad94a5f1d0fc2a0` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:02.042681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1258/2532] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:01 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:02.047000 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3cd1f-f2e5-4a06-a5ee-d5694031ca30 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:02.052293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d420aa2-ca96-4788-8b86-9a311afb3617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b98366f0be45bfbad94a5f1d0fc2a0)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:02.056283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d420aa2-ca96-4788-8b86-9a311afb3617 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:02.058297 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3cd1f-f2e5-4a06-a5ee-d5694031ca30 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:02.064374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378ba9e-7327-4bbc-82dd-fbe2a209f2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:02.065363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378ba9e-7327-4bbc-82dd-fbe2a209f2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:02.065652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378ba9e-7327-4bbc-82dd-fbe2a209f2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:02.065935 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378ba9e-7327-4bbc-82dd-fbe2a209f2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:02.066684 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378ba9e-7327-4bbc-82dd-fbe2a209f2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:02.068371 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378ba9e-7327-4bbc-82dd-fbe2a209f2d5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:02.081696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1259/2533] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:07:02.087218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378ba9e-7327-4bbc-82dd-fbe2a209f2d5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:02.103254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378ba9e-7327-4bbc-82dd-fbe2a209f2d5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:02.110886 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be337775-8e07-4d81-b713-86e614bd8a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:02.111631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be337775-8e07-4d81-b713-86e614bd8a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:02.112475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be337775-8e07-4d81-b713-86e614bd8a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:02.112475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be337775-8e07-4d81-b713-86e614bd8a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:02.113316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be337775-8e07-4d81-b713-86e614bd8a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:02.115222 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be337775-8e07-4d81-b713-86e614bd8a55 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:02.128390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9d420aa2-ca96-4788-8b86-9a311afb3617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b98366f0be45bfbad94a5f1d0fc2a0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:02.129207 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9d420aa2-ca96-4788-8b86-9a311afb3617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b98366f0be45bfbad94a5f1d0fc2a0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:02.134874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1260/2534] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:02.136687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be337775-8e07-4d81-b713-86e614bd8a55 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:02.150461 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9d420aa2-ca96-4788-8b86-9a311afb3617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b98366f0be45bfbad94a5f1d0fc2a0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:02.152608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be337775-8e07-4d81-b713-86e614bd8a55 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:02.159154 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3350002-9c03-4ba5-a62f-8cb8123d327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:02.159846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3350002-9c03-4ba5-a62f-8cb8123d327e admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:02.160132 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3350002-9c03-4ba5-a62f-8cb8123d327e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:02.160412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3350002-9c03-4ba5-a62f-8cb8123d327e admin admin] PATH_INFO: `/v3/projects/346eb7e46ec54cf087d8b39379cf5129/users/9d30603fefe44a0c9dbed8c170071b09/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:02.169007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9d420aa2-ca96-4788-8b86-9a311afb3617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b98366f0be45bfbad94a5f1d0fc2a0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:02.178249 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3350002-9c03-4ba5-a62f-8cb8123d327e admin admin] RBAC: Authorizing `identity:create_grant(project_id=346eb7e46ec54cf087d8b39379cf5129, user_id=9d30603fefe44a0c9dbed8c170071b09, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:02.180986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3350002-9c03-4ba5-a62f-8cb8123d327e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:02.184673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9d420aa2-ca96-4788-8b86-9a311afb3617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b98366f0be45bfbad94a5f1d0fc2a0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:02.206277 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1275/2535] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:02 2022] DELETE /identity/v3/users/90b98366f0be45bfbad94a5f1d0fc2a0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:07:02.211854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7dda8-2a37-41e1-a879-7a77e238ca01 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:02.212238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7dda8-2a37-41e1-a879-7a77e238ca01 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:02.212503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7dda8-2a37-41e1-a879-7a77e238ca01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:02.235587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1261/2536] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:02 2022] PUT /identity/v3/projects/346eb7e46ec54cf087d8b39379cf5129/users/9d30603fefe44a0c9dbed8c170071b09/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:07:02.237286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b44db6-715e-4b06-9174-9bfacdf5f5ee None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:02.237600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b44db6-715e-4b06-9174-9bfacdf5f5ee None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:02.237952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b44db6-715e-4b06-9174-9bfacdf5f5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:02.246317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e5c7dda8-2a37-41e1-a879-7a77e238ca01 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:02.247828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e5c7dda8-2a37-41e1-a879-7a77e238ca01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:02.258867 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-49b44db6-715e-4b06-9174-9bfacdf5f5ee None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:02.260085 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-49b44db6-715e-4b06-9174-9bfacdf5f5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:02.326495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1276/2537] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:07:02.337549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7dda8-2a37-41e1-a879-7a77e238ca01 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:02.338997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e5c7dda8-2a37-41e1-a879-7a77e238ca01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:02.341460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1262/2538] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:02.352589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b44db6-715e-4b06-9174-9bfacdf5f5ee None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:02.359019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-49b44db6-715e-4b06-9174-9bfacdf5f5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:02.362547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:02.387766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:02.435875 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7dda8-2a37-41e1-a879-7a77e238ca01 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:02.440887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba594b-c57a-4df6-b973-b0c3e4e6bd6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:02.441578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba594b-c57a-4df6-b973-b0c3e4e6bd6e service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:02.441887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba594b-c57a-4df6-b973-b0c3e4e6bd6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:02.442168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba594b-c57a-4df6-b973-b0c3e4e6bd6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:02.456226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba594b-c57a-4df6-b973-b0c3e4e6bd6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:02.457674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba594b-c57a-4df6-b973-b0c3e4e6bd6e service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:02.458012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b44db6-715e-4b06-9174-9bfacdf5f5ee None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:02.461871 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b1957-8ecf-4a43-95e0-5884b5e5c7ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:02.462917 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b1957-8ecf-4a43-95e0-5884b5e5c7ec service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:02.463419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b1957-8ecf-4a43-95e0-5884b5e5c7ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:02.463902 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b1957-8ecf-4a43-95e0-5884b5e5c7ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:02.468802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1277/2539] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:07:02.478408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b1957-8ecf-4a43-95e0-5884b5e5c7ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:02.480441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b1957-8ecf-4a43-95e0-5884b5e5c7ec service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:02.504446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1263/2540] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:07:02.882631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba594b-c57a-4df6-b973-b0c3e4e6bd6e service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:02.884294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c8ba594b-c57a-4df6-b973-b0c3e4e6bd6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:02.910278 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba594b-c57a-4df6-b973-b0c3e4e6bd6e service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:02.917152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:02.918104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:02.918554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:02.919059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] PATH_INFO: `/v3/projects/b3c2f5b93f854f32bb6a5c8377539e3f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:02.932424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c2f5b93f854f32bb6a5c8377539e3f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:02.935028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:03.015313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c2f5b93f854f32bb6a5c8377539e3f', 'request_id': 'req-cf34b98d-73ae-4af0-8ee5-9dfa80745321'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.023608 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c2f5b93f854f32bb6a5c8377539e3f', 'request_id': 'req-cf34b98d-73ae-4af0-8ee5-9dfa80745321'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.032402 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c2f5b93f854f32bb6a5c8377539e3f', 'request_id': 'req-cf34b98d-73ae-4af0-8ee5-9dfa80745321'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.057596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] The token cache is being invalidate because project b3c2f5b93f854f32bb6a5c8377539e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:03.057901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.058769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.061429 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1278/2541] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:02 2022] DELETE /identity/v3/projects/b3c2f5b93f854f32bb6a5c8377539e3f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:03.067328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b1957-8ecf-4a43-95e0-5884b5e5c7ec service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:03.083246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b1957-8ecf-4a43-95e0-5884b5e5c7ec service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:03.222746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d9fdf-403d-4bfa-b982-2cdf05c94bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:03.223539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d9fdf-403d-4bfa-b982-2cdf05c94bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:03.224714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d9fdf-403d-4bfa-b982-2cdf05c94bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:03.225047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d9fdf-403d-4bfa-b982-2cdf05c94bac admin admin] PATH_INFO: `/v3/users/a0133f7b523f4db18748158bb74e13c1` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:03.226692 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d9fdf-403d-4bfa-b982-2cdf05c94bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0133f7b523f4db18748158bb74e13c1)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:03.229613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d9fdf-403d-4bfa-b982-2cdf05c94bac admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:03.295376 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3f2d9fdf-403d-4bfa-b982-2cdf05c94bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0133f7b523f4db18748158bb74e13c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.297067 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3f2d9fdf-403d-4bfa-b982-2cdf05c94bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0133f7b523f4db18748158bb74e13c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.307289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3f2d9fdf-403d-4bfa-b982-2cdf05c94bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0133f7b523f4db18748158bb74e13c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.322386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3f2d9fdf-403d-4bfa-b982-2cdf05c94bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0133f7b523f4db18748158bb74e13c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.330226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3f2d9fdf-403d-4bfa-b982-2cdf05c94bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0133f7b523f4db18748158bb74e13c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.337322 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1264/2542] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:03 2022] DELETE /identity/v3/users/a0133f7b523f4db18748158bb74e13c1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:03.630478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:03.632043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:03.647094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34b98d-73ae-4af0-8ee5-9dfa80745321 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:03.724489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:03.725242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:03.725746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:03.727191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] PATH_INFO: `/v3/projects/0b5790108be94ccbad9c9674d04b4123` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:03.729122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b5790108be94ccbad9c9674d04b4123)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:03.733955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:03.797970 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:03.811512 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b5790108be94ccbad9c9674d04b4123', 'request_id': 'req-7c562930-6960-413a-82c4-f3780afcdabd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.818252 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b5790108be94ccbad9c9674d04b4123', 'request_id': 'req-7c562930-6960-413a-82c4-f3780afcdabd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.824949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b5790108be94ccbad9c9674d04b4123', 'request_id': 'req-7c562930-6960-413a-82c4-f3780afcdabd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.841100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] The token cache is being invalidate because project 0b5790108be94ccbad9c9674d04b4123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:03.841660 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.842420 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7c562930-6960-413a-82c4-f3780afcdabd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:03.844821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1279/2543] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:03 2022] DELETE /identity/v3/projects/0b5790108be94ccbad9c9674d04b4123 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:03.849741 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0e0d3-31fc-4be1-885c-05fef7ec5a15 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:03.850056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0e0d3-31fc-4be1-885c-05fef7ec5a15 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:03.850335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0e0d3-31fc-4be1-885c-05fef7ec5a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:03.887842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e1a0e0d3-31fc-4be1-885c-05fef7ec5a15 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:03.889617 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e1a0e0d3-31fc-4be1-885c-05fef7ec5a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:03.930831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:03.972207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1265/2544] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:03.979975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579172a-86c0-4fcb-b20d-87a717fb7a8b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:03.980853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579172a-86c0-4fcb-b20d-87a717fb7a8b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:03.981138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579172a-86c0-4fcb-b20d-87a717fb7a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.002743 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6579172a-86c0-4fcb-b20d-87a717fb7a8b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:04.004343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6579172a-86c0-4fcb-b20d-87a717fb7a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:04.013406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1280/2545] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:04.018389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0e0d3-31fc-4be1-885c-05fef7ec5a15 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.028598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0e0d3-31fc-4be1-885c-05fef7ec5a15 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.035429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c9284-4a70-49f4-860c-b0bd43907bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.036859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9284-4a70-49f4-860c-b0bd43907bca admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.036859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9284-4a70-49f4-860c-b0bd43907bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.037204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9284-4a70-49f4-860c-b0bd43907bca admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.038071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c9284-4a70-49f4-860c-b0bd43907bca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:07:04.038483 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c9284-4a70-49f4-860c-b0bd43907bca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.040362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c9284-4a70-49f4-860c-b0bd43907bca admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:04.052963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1266/2546] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:07:04.057527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579172a-86c0-4fcb-b20d-87a717fb7a8b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.069055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579172a-86c0-4fcb-b20d-87a717fb7a8b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.074524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7560f07-3630-49ba-b4f2-cf461a9e41ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.075176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7560f07-3630-49ba-b4f2-cf461a9e41ae admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.075652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7560f07-3630-49ba-b4f2-cf461a9e41ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.075928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7560f07-3630-49ba-b4f2-cf461a9e41ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.076772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7560f07-3630-49ba-b4f2-cf461a9e41ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.079275 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7560f07-3630-49ba-b4f2-cf461a9e41ae admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:04.113516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1281/2547] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:04 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:04.115952 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c9284-4a70-49f4-860c-b0bd43907bca admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.127750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c9284-4a70-49f4-860c-b0bd43907bca admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.200589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbdb04-a3d2-4f56-965b-7c6e2c2dfd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.201311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbdb04-a3d2-4f56-965b-7c6e2c2dfd04 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.201658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbdb04-a3d2-4f56-965b-7c6e2c2dfd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.202003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbdb04-a3d2-4f56-965b-7c6e2c2dfd04 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.202752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbdb04-a3d2-4f56-965b-7c6e2c2dfd04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.205262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbdb04-a3d2-4f56-965b-7c6e2c2dfd04 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:04.231769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1267/2548] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:04 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:04.237229 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7560f07-3630-49ba-b4f2-cf461a9e41ae admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.249339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7560f07-3630-49ba-b4f2-cf461a9e41ae admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.254874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c54a3-978d-4eb0-a05c-ddab1768aa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.255616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c54a3-978d-4eb0-a05c-ddab1768aa11 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.255997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c54a3-978d-4eb0-a05c-ddab1768aa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.256371 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c54a3-978d-4eb0-a05c-ddab1768aa11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.257184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c54a3-978d-4eb0-a05c-ddab1768aa11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.258935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c54a3-978d-4eb0-a05c-ddab1768aa11 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:04.276662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1282/2549] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:04.277871 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbdb04-a3d2-4f56-965b-7c6e2c2dfd04 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.291304 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbdb04-a3d2-4f56-965b-7c6e2c2dfd04 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.300669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bd817-ec77-4c87-8cdb-54422221ec12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.301315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd817-ec77-4c87-8cdb-54422221ec12 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.301592 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd817-ec77-4c87-8cdb-54422221ec12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.301866 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd817-ec77-4c87-8cdb-54422221ec12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.302566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143bd817-ec77-4c87-8cdb-54422221ec12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.304404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143bd817-ec77-4c87-8cdb-54422221ec12 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:04.315815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1268/2550] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:04.319980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c54a3-978d-4eb0-a05c-ddab1768aa11 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.331598 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c54a3-978d-4eb0-a05c-ddab1768aa11 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.336559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d29071-78d5-45ce-bb7c-527c3fda2cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.337435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29071-78d5-45ce-bb7c-527c3fda2cef admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.337886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29071-78d5-45ce-bb7c-527c3fda2cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.338322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29071-78d5-45ce-bb7c-527c3fda2cef admin admin] PATH_INFO: `/v3/projects/e4dc4cfe58794f23b53124da97c67845/users/c1a10fa76a1d475e8de05a3c847acdc9/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.356048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d29071-78d5-45ce-bb7c-527c3fda2cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4dc4cfe58794f23b53124da97c67845, user_id=c1a10fa76a1d475e8de05a3c847acdc9, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.359070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d29071-78d5-45ce-bb7c-527c3fda2cef admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:04.397517 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1283/2551] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:04 2022] PUT /identity/v3/projects/e4dc4cfe58794f23b53124da97c67845/users/c1a10fa76a1d475e8de05a3c847acdc9/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:07:04.408261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c63c1a-fa56-458f-a897-db4992f686db None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.408602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c63c1a-fa56-458f-a897-db4992f686db None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.408836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c63c1a-fa56-458f-a897-db4992f686db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.443635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e3c63c1a-fa56-458f-a897-db4992f686db None None] MFA Rules not processed for user `c1a10fa76a1d475e8de05a3c847acdc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:04.444954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e3c63c1a-fa56-458f-a897-db4992f686db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:04.529370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1269/2552] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:04 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:04.538194 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d29071-78d5-45ce-bb7c-527c3fda2cef admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.611653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d29071-78d5-45ce-bb7c-527c3fda2cef admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.616508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087aea1-e1a5-40cb-b574-ab208675202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.617128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087aea1-e1a5-40cb-b574-ab208675202d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.617412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087aea1-e1a5-40cb-b574-ab208675202d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.617690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087aea1-e1a5-40cb-b574-ab208675202d admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.618378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087aea1-e1a5-40cb-b574-ab208675202d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.620334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087aea1-e1a5-40cb-b574-ab208675202d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:04.651049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1284/2553] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:04 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:04.652606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c63c1a-fa56-458f-a897-db4992f686db None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.664645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c63c1a-fa56-458f-a897-db4992f686db None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.695361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:04.731193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5c80c-890d-481d-a699-ef5fca89ed84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.731737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5c80c-890d-481d-a699-ef5fca89ed84 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.732019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5c80c-890d-481d-a699-ef5fca89ed84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.732291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5c80c-890d-481d-a699-ef5fca89ed84 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.732967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5c80c-890d-481d-a699-ef5fca89ed84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.735624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5c80c-890d-481d-a699-ef5fca89ed84 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:04.761858 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1270/2554] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:04 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:04.765865 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087aea1-e1a5-40cb-b574-ab208675202d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.776601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087aea1-e1a5-40cb-b574-ab208675202d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.781328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2f8a5-9bb2-4720-b69d-0feabbd8f11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.783674 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2f8a5-9bb2-4720-b69d-0feabbd8f11a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.783820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2f8a5-9bb2-4720-b69d-0feabbd8f11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.784005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2f8a5-9bb2-4720-b69d-0feabbd8f11a admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.784708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2f8a5-9bb2-4720-b69d-0feabbd8f11a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.786601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2f8a5-9bb2-4720-b69d-0feabbd8f11a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:04.799201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1285/2555] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:04.803332 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5c80c-890d-481d-a699-ef5fca89ed84 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.813930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5c80c-890d-481d-a699-ef5fca89ed84 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.818777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f8285-4be4-4dc9-8867-7a50a80fb2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.819658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f8285-4be4-4dc9-8867-7a50a80fb2d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.819938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f8285-4be4-4dc9-8867-7a50a80fb2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.820341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f8285-4be4-4dc9-8867-7a50a80fb2d0 admin admin] PATH_INFO: `/v3/projects/d75afb7424fb4af391da8b2cc8bab98a/users/e0142b933a7f40a7adc971de3fc4ea75/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.836705 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f8285-4be4-4dc9-8867-7a50a80fb2d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75afb7424fb4af391da8b2cc8bab98a, user_id=e0142b933a7f40a7adc971de3fc4ea75, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.839412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f8285-4be4-4dc9-8867-7a50a80fb2d0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:04.855129 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1271/2556] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:04 2022] PUT /identity/v3/projects/d75afb7424fb4af391da8b2cc8bab98a/users/e0142b933a7f40a7adc971de3fc4ea75/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:07:04.859218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2f8a5-9bb2-4720-b69d-0feabbd8f11a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.871465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2f8a5-9bb2-4720-b69d-0feabbd8f11a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.916771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:04.946870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc742b-8674-4659-b442-f4fbde9ac1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.947596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc742b-8674-4659-b442-f4fbde9ac1cd admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.948094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc742b-8674-4659-b442-f4fbde9ac1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.948449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc742b-8674-4659-b442-f4fbde9ac1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.949234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc742b-8674-4659-b442-f4fbde9ac1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.950979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc742b-8674-4659-b442-f4fbde9ac1cd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:04.966355 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1286/2557] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:04.967766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f8285-4be4-4dc9-8867-7a50a80fb2d0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:04.979315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f8285-4be4-4dc9-8867-7a50a80fb2d0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:04.988589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5fba7-a8fd-4fd7-988b-fc5171864c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:04.989579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5fba7-a8fd-4fd7-988b-fc5171864c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:04.989872 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5fba7-a8fd-4fd7-988b-fc5171864c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:04.990159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5fba7-a8fd-4fd7-988b-fc5171864c6a admin admin] PATH_INFO: `/v3/projects/d75afb7424fb4af391da8b2cc8bab98a/users/e0142b933a7f40a7adc971de3fc4ea75/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:04.994495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5fba7-a8fd-4fd7-988b-fc5171864c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75afb7424fb4af391da8b2cc8bab98a, user_id=e0142b933a7f40a7adc971de3fc4ea75, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:04.997514 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5fba7-a8fd-4fd7-988b-fc5171864c6a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:05.010356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1272/2558] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:04 2022] PUT /identity/v3/projects/d75afb7424fb4af391da8b2cc8bab98a/users/e0142b933a7f40a7adc971de3fc4ea75/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:05.022086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07df66-8b96-4484-bfab-80ef8eab0bba None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:05.023727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07df66-8b96-4484-bfab-80ef8eab0bba None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:05.024010 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07df66-8b96-4484-bfab-80ef8eab0bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:05.056467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ec07df66-8b96-4484-bfab-80ef8eab0bba None None] MFA Rules not processed for user `e0142b933a7f40a7adc971de3fc4ea75`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:05.057814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ec07df66-8b96-4484-bfab-80ef8eab0bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:05.142686 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1287/2559] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:05.150508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5fba7-a8fd-4fd7-988b-fc5171864c6a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:05.152847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-14c5fba7-a8fd-4fd7-988b-fc5171864c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:05.241504 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5fba7-a8fd-4fd7-988b-fc5171864c6a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:05.247725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14dede-50f9-4ebc-b862-7e4573ef620f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:05.248955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14dede-50f9-4ebc-b862-7e4573ef620f service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:05.248955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14dede-50f9-4ebc-b862-7e4573ef620f service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:05.249142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14dede-50f9-4ebc-b862-7e4573ef620f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:05.269514 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14dede-50f9-4ebc-b862-7e4573ef620f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:05.271496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14dede-50f9-4ebc-b862-7e4573ef620f service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:05.287365 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1273/2560] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:05.378773 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07df66-8b96-4484-bfab-80ef8eab0bba None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:05.390144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07df66-8b96-4484-bfab-80ef8eab0bba None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:05.395388 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd731d-a4e3-41a2-9d3b-1c254f20402c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:05.396128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd731d-a4e3-41a2-9d3b-1c254f20402c service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:05.396512 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd731d-a4e3-41a2-9d3b-1c254f20402c service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:05.396889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd731d-a4e3-41a2-9d3b-1c254f20402c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:05.464944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd731d-a4e3-41a2-9d3b-1c254f20402c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:05.466504 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd731d-a4e3-41a2-9d3b-1c254f20402c service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:05.478594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1288/2561] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:05 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:07.801093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957eeda8-802b-47a5-8919-683d7ab23d2d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:07.801578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957eeda8-802b-47a5-8919-683d7ab23d2d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:07.801578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957eeda8-802b-47a5-8919-683d7ab23d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:07.831235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-957eeda8-802b-47a5-8919-683d7ab23d2d None None] MFA Rules not processed for user `9d30603fefe44a0c9dbed8c170071b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:07.832480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-957eeda8-802b-47a5-8919-683d7ab23d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:07.910092 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1274/2562] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:07.915040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd731d-a4e3-41a2-9d3b-1c254f20402c service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:07.916592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e1bd731d-a4e3-41a2-9d3b-1c254f20402c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:07.985218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:07.994995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd731d-a4e3-41a2-9d3b-1c254f20402c service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:07.999746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e6860-2440-4a8c-a1c1-79683732bb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:08.000995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e6860-2440-4a8c-a1c1-79683732bb70 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:08.001428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e6860-2440-4a8c-a1c1-79683732bb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:08.001720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e6860-2440-4a8c-a1c1-79683732bb70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:08.002404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e6860-2440-4a8c-a1c1-79683732bb70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:08.004444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e6860-2440-4a8c-a1c1-79683732bb70 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:08.041771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1289/2563] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:07 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:08.046130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957eeda8-802b-47a5-8919-683d7ab23d2d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:08.058482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957eeda8-802b-47a5-8919-683d7ab23d2d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:08.070648 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:08.131653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba9a90-2cec-48d6-ab93-f483f1c2cee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:08.132370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba9a90-2cec-48d6-ab93-f483f1c2cee8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:08.132803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba9a90-2cec-48d6-ab93-f483f1c2cee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:08.133182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba9a90-2cec-48d6-ab93-f483f1c2cee8 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:08.134161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba9a90-2cec-48d6-ab93-f483f1c2cee8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:08.136388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba9a90-2cec-48d6-ab93-f483f1c2cee8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:08.161874 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1275/2564] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:08 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:08.165914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e6860-2440-4a8c-a1c1-79683732bb70 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:08.181025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e6860-2440-4a8c-a1c1-79683732bb70 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:08.186157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a77d56-f963-4fd9-a79b-eb4dde2e4be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:08.186891 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a77d56-f963-4fd9-a79b-eb4dde2e4be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:08.187325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a77d56-f963-4fd9-a79b-eb4dde2e4be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:08.187605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a77d56-f963-4fd9-a79b-eb4dde2e4be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:08.188689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a77d56-f963-4fd9-a79b-eb4dde2e4be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:08.190405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a77d56-f963-4fd9-a79b-eb4dde2e4be9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:08.204613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1290/2565] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:08.208945 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba9a90-2cec-48d6-ab93-f483f1c2cee8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:08.221881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba9a90-2cec-48d6-ab93-f483f1c2cee8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:08.226842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad01ea-10f5-4dc9-8696-5c043a5fb17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:08.227530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad01ea-10f5-4dc9-8696-5c043a5fb17d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:08.228516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad01ea-10f5-4dc9-8696-5c043a5fb17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:08.228803 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad01ea-10f5-4dc9-8696-5c043a5fb17d admin admin] PATH_INFO: `/v3/projects/cdf6f41af5c34ec9b449157e36c4b372/users/c992bd261bdb418cb1a4b604a8e1bd8d/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:08.245440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad01ea-10f5-4dc9-8696-5c043a5fb17d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf6f41af5c34ec9b449157e36c4b372, user_id=c992bd261bdb418cb1a4b604a8e1bd8d, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:08.248286 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad01ea-10f5-4dc9-8696-5c043a5fb17d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:08.263675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1276/2566] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:08 2022] PUT /identity/v3/projects/cdf6f41af5c34ec9b449157e36c4b372/users/c992bd261bdb418cb1a4b604a8e1bd8d/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:08.268073 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a77d56-f963-4fd9-a79b-eb4dde2e4be9 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:08.279376 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a77d56-f963-4fd9-a79b-eb4dde2e4be9 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:08.347024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53da414-c756-4e6a-9b73-5d148085847b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:08.347659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53da414-c756-4e6a-9b73-5d148085847b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:08.347950 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53da414-c756-4e6a-9b73-5d148085847b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:08.348231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53da414-c756-4e6a-9b73-5d148085847b admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:08.348934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53da414-c756-4e6a-9b73-5d148085847b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:08.350552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53da414-c756-4e6a-9b73-5d148085847b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:08.362543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1291/2567] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:08.366842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad01ea-10f5-4dc9-8696-5c043a5fb17d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:08.377896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad01ea-10f5-4dc9-8696-5c043a5fb17d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:08.382563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b0124-acf0-4810-9677-ecc66a0f9c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:08.383267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b0124-acf0-4810-9677-ecc66a0f9c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:08.383547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b0124-acf0-4810-9677-ecc66a0f9c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:08.383822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b0124-acf0-4810-9677-ecc66a0f9c85 admin admin] PATH_INFO: `/v3/projects/cdf6f41af5c34ec9b449157e36c4b372/users/c992bd261bdb418cb1a4b604a8e1bd8d/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:08.385787 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b0124-acf0-4810-9677-ecc66a0f9c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf6f41af5c34ec9b449157e36c4b372, user_id=c992bd261bdb418cb1a4b604a8e1bd8d, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:08.388416 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b0124-acf0-4810-9677-ecc66a0f9c85 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:08.400092 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1277/2568] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:08 2022] PUT /identity/v3/projects/cdf6f41af5c34ec9b449157e36c4b372/users/c992bd261bdb418cb1a4b604a8e1bd8d/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:07:08.579103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53da414-c756-4e6a-9b73-5d148085847b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:08.596062 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53da414-c756-4e6a-9b73-5d148085847b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:08.673583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:08.674238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:08.674519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:08.674806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] PATH_INFO: `/v3/users/c1a10fa76a1d475e8de05a3c847acdc9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:08.676661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1a10fa76a1d475e8de05a3c847acdc9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:08.678461 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:08.742071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1a10fa76a1d475e8de05a3c847acdc9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:08.742964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1a10fa76a1d475e8de05a3c847acdc9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:08.790016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1a10fa76a1d475e8de05a3c847acdc9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:08.797886 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:08.814900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1a10fa76a1d475e8de05a3c847acdc9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:08.827452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1a10fa76a1d475e8de05a3c847acdc9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:08.838256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1292/2569] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:08 2022] DELETE /identity/v3/users/c1a10fa76a1d475e8de05a3c847acdc9 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:08.842458 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43105-6a7e-45e9-b630-d6c3f2ed94ba None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:08.843017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43105-6a7e-45e9-b630-d6c3f2ed94ba None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:08.843017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43105-6a7e-45e9-b630-d6c3f2ed94ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:08.864329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b3b43105-6a7e-45e9-b630-d6c3f2ed94ba None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:08.865567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b3b43105-6a7e-45e9-b630-d6c3f2ed94ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:08.967472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1278/2570] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:08.978514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:08.980284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:09.050647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71261c3-c561-4bab-82e9-a3dd737ad6fe admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:09.056918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831cc5f-5943-4400-b2be-b731e8ba2451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:09.057723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831cc5f-5943-4400-b2be-b731e8ba2451 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:09.058185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831cc5f-5943-4400-b2be-b731e8ba2451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:09.058453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831cc5f-5943-4400-b2be-b731e8ba2451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:09.070489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831cc5f-5943-4400-b2be-b731e8ba2451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:09.072210 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831cc5f-5943-4400-b2be-b731e8ba2451 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:09.086152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1293/2571] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:09.179645 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684882f7-eeb5-4aec-8775-26c4fc2ce532 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:09.179981 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684882f7-eeb5-4aec-8775-26c4fc2ce532 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:09.180248 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684882f7-eeb5-4aec-8775-26c4fc2ce532 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:09.181879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1279/2572] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jul 28 04:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 04:07:09.213839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48204d8-0540-458f-b6ca-caa1fa877a8b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:09.214316 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48204d8-0540-458f-b6ca-caa1fa877a8b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:09.214604 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48204d8-0540-458f-b6ca-caa1fa877a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:09.265756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d48204d8-0540-458f-b6ca-caa1fa877a8b None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:09.267555 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d48204d8-0540-458f-b6ca-caa1fa877a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:09.480844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1294/2573] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jul 28 04:07:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:09.496960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684882f7-eeb5-4aec-8775-26c4fc2ce532 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:09.498768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-684882f7-eeb5-4aec-8775-26c4fc2ce532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:09.603574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48204d8-0540-458f-b6ca-caa1fa877a8b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:09.605179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d48204d8-0540-458f-b6ca-caa1fa877a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:09.610853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684882f7-eeb5-4aec-8775-26c4fc2ce532 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:09.617272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f0522-5c3f-45da-9f73-318b865e9394 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8967641dd624dacb5ff12c8e22599bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:09.618253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f0522-5c3f-45da-9f73-318b865e9394 service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:09.619130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f0522-5c3f-45da-9f73-318b865e9394 service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:09.619687 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f0522-5c3f-45da-9f73-318b865e9394 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:09.634214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f0522-5c3f-45da-9f73-318b865e9394 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:09.636334 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f0522-5c3f-45da-9f73-318b865e9394 service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:09.644430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48204d8-0540-458f-b6ca-caa1fa877a8b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:09.655152 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1280/2574] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:09.656559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:09.657206 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:09.657501 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:09.657784 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] PATH_INFO: `/v3/projects/e4dc4cfe58794f23b53124da97c67845` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:09.660622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4dc4cfe58794f23b53124da97c67845)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:09.662927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:09.756466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b870a33-587f-48fb-9509-ee933d355a2b None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:09.756832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b870a33-587f-48fb-9509-ee933d355a2b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:09.757093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b870a33-587f-48fb-9509-ee933d355a2b None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:09.758565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1281/2575] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jul 28 04:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 04:07:09.764310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528660f1-410d-41f2-8083-d821a48b96f7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:09.764984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528660f1-410d-41f2-8083-d821a48b96f7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:09.765315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528660f1-410d-41f2-8083-d821a48b96f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:09.771842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4dc4cfe58794f23b53124da97c67845', 'request_id': 'req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:09.778814 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4dc4cfe58794f23b53124da97c67845', 'request_id': 'req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:09.787779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4dc4cfe58794f23b53124da97c67845', 'request_id': 'req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:09.812385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] The token cache is being invalidate because project e4dc4cfe58794f23b53124da97c67845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:09.813633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:09.815634 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:09.816821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-528660f1-410d-41f2-8083-d821a48b96f7 None None] MFA Rules not processed for user `6e31408bfea4412d8f68aea9b3c7528d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:09.820742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1295/2576] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:09 2022] DELETE /identity/v3/projects/e4dc4cfe58794f23b53124da97c67845 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:09.824338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-528660f1-410d-41f2-8083-d821a48b96f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:09.825442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:09.830525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:09.835519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:09.842907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:09.945182 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2861876-6cf2-4ccb-9e05-3a84fbe5fa82 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:09.949867 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1282/2577] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jul 28 04:07:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:07:09.956447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747f3b2-8e45-4acb-be9e-e8b0c74d81b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:09.957147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747f3b2-8e45-4acb-be9e-e8b0c74d81b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:09.957442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747f3b2-8e45-4acb-be9e-e8b0c74d81b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:09.957729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747f3b2-8e45-4acb-be9e-e8b0c74d81b3 admin admin] PATH_INFO: `/v3/users/e0142b933a7f40a7adc971de3fc4ea75` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:09.959136 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747f3b2-8e45-4acb-be9e-e8b0c74d81b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0142b933a7f40a7adc971de3fc4ea75)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:09.961187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747f3b2-8e45-4acb-be9e-e8b0c74d81b3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:09.962924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528660f1-410d-41f2-8083-d821a48b96f7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:09.964708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-528660f1-410d-41f2-8083-d821a48b96f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:09.991441 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528660f1-410d-41f2-8083-d821a48b96f7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:10.002423 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3919d8-18b2-4081-b22a-4985460a4cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:10.007698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3919d8-18b2-4081-b22a-4985460a4cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:10.007943 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3919d8-18b2-4081-b22a-4985460a4cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:10.008238 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3919d8-18b2-4081-b22a-4985460a4cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:10.027253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3919d8-18b2-4081-b22a-4985460a4cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:10.027459 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d747f3b2-8e45-4acb-be9e-e8b0c74d81b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0142b933a7f40a7adc971de3fc4ea75'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:10.028024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d747f3b2-8e45-4acb-be9e-e8b0c74d81b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0142b933a7f40a7adc971de3fc4ea75'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:10.028836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3919d8-18b2-4081-b22a-4985460a4cf6 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:10.044150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d747f3b2-8e45-4acb-be9e-e8b0c74d81b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0142b933a7f40a7adc971de3fc4ea75'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:10.046937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1283/2578] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:07:10.052785 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d747f3b2-8e45-4acb-be9e-e8b0c74d81b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0142b933a7f40a7adc971de3fc4ea75'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:10.069777 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d747f3b2-8e45-4acb-be9e-e8b0c74d81b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0142b933a7f40a7adc971de3fc4ea75'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:10.082506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1296/2579] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:09 2022] DELETE /identity/v3/users/e0142b933a7f40a7adc971de3fc4ea75 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:10.190094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb69022-0798-4421-b703-a2976c5039ed None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:10.190415 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb69022-0798-4421-b703-a2976c5039ed None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:10.191079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb69022-0798-4421-b703-a2976c5039ed None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:10.192086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1284/2580] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jul 28 04:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 04:07:10.202671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa5ca-9843-4b51-8fda-84f52144c4d0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:10.203164 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa5ca-9843-4b51-8fda-84f52144c4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:10.203751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa5ca-9843-4b51-8fda-84f52144c4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:10.255937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e00fa5ca-9843-4b51-8fda-84f52144c4d0 None None] MFA Rules not processed for user `162ba16d643747c287bfe8a8e59df235`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:10.260716 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e00fa5ca-9843-4b51-8fda-84f52144c4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:10.385579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1297/2581] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jul 28 04:07:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:10.398118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb69022-0798-4421-b703-a2976c5039ed None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:10.401122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0cb69022-0798-4421-b703-a2976c5039ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:10.510601 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb69022-0798-4421-b703-a2976c5039ed None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:10.519478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5c60f-f478-4f00-90c1-bd7a5b052f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:10.520819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5c60f-f478-4f00-90c1-bd7a5b052f77 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:10.521443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5c60f-f478-4f00-90c1-bd7a5b052f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:10.521903 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5c60f-f478-4f00-90c1-bd7a5b052f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:10.544992 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5c60f-f478-4f00-90c1-bd7a5b052f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:10.546668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5c60f-f478-4f00-90c1-bd7a5b052f77 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:10.589688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1285/2582] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:10.908016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fa5ca-9843-4b51-8fda-84f52144c4d0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:10.909509 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e00fa5ca-9843-4b51-8fda-84f52144c4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:10.967719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:11.050230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fa5ca-9843-4b51-8fda-84f52144c4d0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:11.055628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:11.056307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:11.056592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:11.056868 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] PATH_INFO: `/v3/projects/d75afb7424fb4af391da8b2cc8bab98a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:11.059637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75afb7424fb4af391da8b2cc8bab98a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:11.061675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:11.173358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75afb7424fb4af391da8b2cc8bab98a', 'request_id': 'req-96576224-186c-4d9d-9c4d-96a9a6b8c22b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:11.183226 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75afb7424fb4af391da8b2cc8bab98a', 'request_id': 'req-96576224-186c-4d9d-9c4d-96a9a6b8c22b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:11.194108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75afb7424fb4af391da8b2cc8bab98a', 'request_id': 'req-96576224-186c-4d9d-9c4d-96a9a6b8c22b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:11.215221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] The token cache is being invalidate because project d75afb7424fb4af391da8b2cc8bab98a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:11.215676 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:11.216353 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-96576224-186c-4d9d-9c4d-96a9a6b8c22b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:11.218261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1298/2583] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:10 2022] DELETE /identity/v3/projects/d75afb7424fb4af391da8b2cc8bab98a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:11.228968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953be5a5-295d-4927-b2d7-3428e01b6f9f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:11.229655 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953be5a5-295d-4927-b2d7-3428e01b6f9f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:11.230146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953be5a5-295d-4927-b2d7-3428e01b6f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:11.264142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-953be5a5-295d-4927-b2d7-3428e01b6f9f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:11.266516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-953be5a5-295d-4927-b2d7-3428e01b6f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:11.392578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1286/2584] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:11.402406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccda4fc-2a75-4cf5-bf98-b30e6e9b9a1a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:11.402821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccda4fc-2a75-4cf5-bf98-b30e6e9b9a1a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:11.403212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccda4fc-2a75-4cf5-bf98-b30e6e9b9a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:11.430026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4ccda4fc-2a75-4cf5-bf98-b30e6e9b9a1a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:11.439121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4ccda4fc-2a75-4cf5-bf98-b30e6e9b9a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:11.463070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1299/2585] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:11.467125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953be5a5-295d-4927-b2d7-3428e01b6f9f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:11.478069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953be5a5-295d-4927-b2d7-3428e01b6f9f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:11.488841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864421fb-e19c-4c2b-afc6-0d5ddf49d7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:11.489934 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864421fb-e19c-4c2b-afc6-0d5ddf49d7cf admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:11.491043 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864421fb-e19c-4c2b-afc6-0d5ddf49d7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:11.491517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864421fb-e19c-4c2b-afc6-0d5ddf49d7cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:11.492425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864421fb-e19c-4c2b-afc6-0d5ddf49d7cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:07:11.493006 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864421fb-e19c-4c2b-afc6-0d5ddf49d7cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:11.495023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864421fb-e19c-4c2b-afc6-0d5ddf49d7cf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:11.519150 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1287/2586] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:07:11.523405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccda4fc-2a75-4cf5-bf98-b30e6e9b9a1a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:11.534357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccda4fc-2a75-4cf5-bf98-b30e6e9b9a1a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:11.541618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9cf4f-3199-4418-abd9-5c6338f4b6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:11.543595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9cf4f-3199-4418-abd9-5c6338f4b6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:11.543595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9cf4f-3199-4418-abd9-5c6338f4b6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:11.543595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9cf4f-3199-4418-abd9-5c6338f4b6d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:11.543595 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9cf4f-3199-4418-abd9-5c6338f4b6d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:11.545503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9cf4f-3199-4418-abd9-5c6338f4b6d0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:11.601385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1300/2587] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:11 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:11.606326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864421fb-e19c-4c2b-afc6-0d5ddf49d7cf admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:11.627538 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864421fb-e19c-4c2b-afc6-0d5ddf49d7cf admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:11.697251 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:11.742651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37849ba3-e039-4111-90fb-6dd8fb8980be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:11.743307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37849ba3-e039-4111-90fb-6dd8fb8980be admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:11.743591 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37849ba3-e039-4111-90fb-6dd8fb8980be admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:11.743885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37849ba3-e039-4111-90fb-6dd8fb8980be admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:11.744756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37849ba3-e039-4111-90fb-6dd8fb8980be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:11.746756 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37849ba3-e039-4111-90fb-6dd8fb8980be admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:11.797203 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1288/2588] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:11 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:11.801561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9cf4f-3199-4418-abd9-5c6338f4b6d0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:11.817887 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9cf4f-3199-4418-abd9-5c6338f4b6d0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:11.833817 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd66a55-1c7a-4f2d-b8e0-54715911be96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:11.834489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd66a55-1c7a-4f2d-b8e0-54715911be96 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:11.834988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd66a55-1c7a-4f2d-b8e0-54715911be96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:11.835271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd66a55-1c7a-4f2d-b8e0-54715911be96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:11.836138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd66a55-1c7a-4f2d-b8e0-54715911be96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:11.837840 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd66a55-1c7a-4f2d-b8e0-54715911be96 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:11.865510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37849ba3-e039-4111-90fb-6dd8fb8980be admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:11.872214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1301/2589] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:11.879318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37849ba3-e039-4111-90fb-6dd8fb8980be admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:11.888738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76d116-3f66-42ad-8762-195e10d0064f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:11.889654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76d116-3f66-42ad-8762-195e10d0064f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:11.890113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76d116-3f66-42ad-8762-195e10d0064f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:11.891045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76d116-3f66-42ad-8762-195e10d0064f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:11.891916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76d116-3f66-42ad-8762-195e10d0064f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:11.895465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76d116-3f66-42ad-8762-195e10d0064f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:11.915580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1289/2590] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:11.919180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd66a55-1c7a-4f2d-b8e0-54715911be96 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:11.943090 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd66a55-1c7a-4f2d-b8e0-54715911be96 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:11.948342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a6380-cb74-4ab5-be9b-5d3ae88310dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:11.949708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a6380-cb74-4ab5-be9b-5d3ae88310dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:11.950188 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a6380-cb74-4ab5-be9b-5d3ae88310dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:11.950643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a6380-cb74-4ab5-be9b-5d3ae88310dd admin admin] PATH_INFO: `/v3/projects/855a939a8fe8445d9bb11f6959451f0a/users/f33a0683b1dd44cd9164f5636f8f1006/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:12.023265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a6380-cb74-4ab5-be9b-5d3ae88310dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=855a939a8fe8445d9bb11f6959451f0a, user_id=f33a0683b1dd44cd9164f5636f8f1006, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:12.026224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a6380-cb74-4ab5-be9b-5d3ae88310dd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:12.066678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1302/2591] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:11 2022] PUT /identity/v3/projects/855a939a8fe8445d9bb11f6959451f0a/users/f33a0683b1dd44cd9164f5636f8f1006/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:12.080957 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d45487-0a07-489a-bba3-f0bf4e272f9a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:12.081579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d45487-0a07-489a-bba3-f0bf4e272f9a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:12.082045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d45487-0a07-489a-bba3-f0bf4e272f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:12.148753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-24d45487-0a07-489a-bba3-f0bf4e272f9a None None] MFA Rules not processed for user `f33a0683b1dd44cd9164f5636f8f1006`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:12.151797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-24d45487-0a07-489a-bba3-f0bf4e272f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:12.292103 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1290/2592] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:12.297603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a6380-cb74-4ab5-be9b-5d3ae88310dd admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:12.393085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:12.433740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a6380-cb74-4ab5-be9b-5d3ae88310dd admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:12.439703 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09045b1-1bae-4e31-9556-c83368f82e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:12.440350 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09045b1-1bae-4e31-9556-c83368f82e1c admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:12.440637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09045b1-1bae-4e31-9556-c83368f82e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:12.440923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09045b1-1bae-4e31-9556-c83368f82e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:12.441590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09045b1-1bae-4e31-9556-c83368f82e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:12.443574 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09045b1-1bae-4e31-9556-c83368f82e1c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:12.487506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1303/2593] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:12 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:07:12.488055 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d45487-0a07-489a-bba3-f0bf4e272f9a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:12.502761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d45487-0a07-489a-bba3-f0bf4e272f9a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:12.624267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713c8c-d9fa-4f2f-a036-1483f9f6e89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:12.625038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713c8c-d9fa-4f2f-a036-1483f9f6e89f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:12.626085 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713c8c-d9fa-4f2f-a036-1483f9f6e89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:12.626615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713c8c-d9fa-4f2f-a036-1483f9f6e89f admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:12.626906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d713c8c-d9fa-4f2f-a036-1483f9f6e89f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:12.630419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d713c8c-d9fa-4f2f-a036-1483f9f6e89f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:12.668279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09045b1-1bae-4e31-9556-c83368f82e1c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:12.678647 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1291/2594] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:12 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:07:12.681490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09045b1-1bae-4e31-9556-c83368f82e1c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:12.686981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddb4ea-a444-4d3f-96d8-a08065d39c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:12.688437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddb4ea-a444-4d3f-96d8-a08065d39c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:12.689074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddb4ea-a444-4d3f-96d8-a08065d39c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:12.689074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddb4ea-a444-4d3f-96d8-a08065d39c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:12.689790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddb4ea-a444-4d3f-96d8-a08065d39c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:12.691689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddb4ea-a444-4d3f-96d8-a08065d39c7f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:12.714426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713c8c-d9fa-4f2f-a036-1483f9f6e89f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:12.718973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1304/2595] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:12.732600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713c8c-d9fa-4f2f-a036-1483f9f6e89f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:12.741223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035e38e-957b-4a61-835f-17850f165dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:12.742708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035e38e-957b-4a61-835f-17850f165dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:12.743058 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035e38e-957b-4a61-835f-17850f165dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:12.744821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035e38e-957b-4a61-835f-17850f165dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:12.746181 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035e38e-957b-4a61-835f-17850f165dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:12.749210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035e38e-957b-4a61-835f-17850f165dc3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:12.768079 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1292/2596] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:12.769142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddb4ea-a444-4d3f-96d8-a08065d39c7f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:12.783570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddb4ea-a444-4d3f-96d8-a08065d39c7f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:12.789913 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa625-b544-48d6-8105-710f41661dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:12.790591 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa625-b544-48d6-8105-710f41661dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:12.790935 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa625-b544-48d6-8105-710f41661dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:12.791218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa625-b544-48d6-8105-710f41661dfb admin admin] PATH_INFO: `/v3/projects/ca88928b7ccc446a8380804c4860b113/users/e70f77bdf6634aa3b3de5b075af6f587/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:12.824170 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fa625-b544-48d6-8105-710f41661dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca88928b7ccc446a8380804c4860b113, user_id=e70f77bdf6634aa3b3de5b075af6f587, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:12.839242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fa625-b544-48d6-8105-710f41661dfb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:12.861231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1305/2597] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:12 2022] PUT /identity/v3/projects/ca88928b7ccc446a8380804c4860b113/users/e70f77bdf6634aa3b3de5b075af6f587/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:12.871477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a9fe6-7728-453c-8b24-31f946b117ed None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:12.871771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a9fe6-7728-453c-8b24-31f946b117ed None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:12.872054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a9fe6-7728-453c-8b24-31f946b117ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:12.913387 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-022a9fe6-7728-453c-8b24-31f946b117ed None None] MFA Rules not processed for user `e70f77bdf6634aa3b3de5b075af6f587`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:12.915568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-022a9fe6-7728-453c-8b24-31f946b117ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:12.931137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:13.047310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1293/2598] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:13.052131 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa625-b544-48d6-8105-710f41661dfb admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:13.192416 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa625-b544-48d6-8105-710f41661dfb admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:13.206508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09686767-07c7-48a6-bc20-d51663234fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:13.207709 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686767-07c7-48a6-bc20-d51663234fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:13.208002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686767-07c7-48a6-bc20-d51663234fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:13.208282 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686767-07c7-48a6-bc20-d51663234fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:13.209060 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09686767-07c7-48a6-bc20-d51663234fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:13.211076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09686767-07c7-48a6-bc20-d51663234fb1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:13.262712 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1306/2599] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:13 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:13.269287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a9fe6-7728-453c-8b24-31f946b117ed None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:13.292938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a9fe6-7728-453c-8b24-31f946b117ed None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:13.413403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc47c4-e831-4a77-915f-f67f0ec6af3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:13.414475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc47c4-e831-4a77-915f-f67f0ec6af3d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:13.414948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc47c4-e831-4a77-915f-f67f0ec6af3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:13.415393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc47c4-e831-4a77-915f-f67f0ec6af3d admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:13.416221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc47c4-e831-4a77-915f-f67f0ec6af3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:13.418351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc47c4-e831-4a77-915f-f67f0ec6af3d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:13.456936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1294/2600] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:13 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:13.457243 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09686767-07c7-48a6-bc20-d51663234fb1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:13.471114 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09686767-07c7-48a6-bc20-d51663234fb1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:13.476255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77d386-25e7-489b-b423-c71cae41f3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:13.476932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77d386-25e7-489b-b423-c71cae41f3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:13.477222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77d386-25e7-489b-b423-c71cae41f3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:13.477499 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77d386-25e7-489b-b423-c71cae41f3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:13.478403 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77d386-25e7-489b-b423-c71cae41f3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:13.480081 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77d386-25e7-489b-b423-c71cae41f3f3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:13.497754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1307/2601] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:13.499060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc47c4-e831-4a77-915f-f67f0ec6af3d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:13.510414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc47c4-e831-4a77-915f-f67f0ec6af3d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:13.516248 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828483d-73e6-4f7e-b5a4-17c6916e8c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:13.516948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828483d-73e6-4f7e-b5a4-17c6916e8c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:13.517235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828483d-73e6-4f7e-b5a4-17c6916e8c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:13.517636 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828483d-73e6-4f7e-b5a4-17c6916e8c70 admin admin] PATH_INFO: `/v3/projects/db029da6fcef496ba63018aa203aa3c0/users/14c47af8f963444f93307b6c6bd00a52/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:13.539887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828483d-73e6-4f7e-b5a4-17c6916e8c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db029da6fcef496ba63018aa203aa3c0, user_id=14c47af8f963444f93307b6c6bd00a52, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:13.543992 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828483d-73e6-4f7e-b5a4-17c6916e8c70 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:13.584958 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77d386-25e7-489b-b423-c71cae41f3f3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:13.585891 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1295/2602] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:13 2022] PUT /identity/v3/projects/db029da6fcef496ba63018aa203aa3c0/users/14c47af8f963444f93307b6c6bd00a52/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:13.596427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77d386-25e7-489b-b423-c71cae41f3f3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:13.686000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211a95d-e8b6-49e5-82f1-6c6f0b7c9df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:13.686901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a95d-e8b6-49e5-82f1-6c6f0b7c9df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:13.687310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a95d-e8b6-49e5-82f1-6c6f0b7c9df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:13.688981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a95d-e8b6-49e5-82f1-6c6f0b7c9df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:13.689815 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211a95d-e8b6-49e5-82f1-6c6f0b7c9df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:13.691797 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211a95d-e8b6-49e5-82f1-6c6f0b7c9df0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:13.708910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1308/2603] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:13.710192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828483d-73e6-4f7e-b5a4-17c6916e8c70 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:13.721955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828483d-73e6-4f7e-b5a4-17c6916e8c70 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:13.726812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141272f-3949-446f-a514-15a3d7dc8f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:13.727486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141272f-3949-446f-a514-15a3d7dc8f7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:13.727971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141272f-3949-446f-a514-15a3d7dc8f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:13.728288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141272f-3949-446f-a514-15a3d7dc8f7d admin admin] PATH_INFO: `/v3/projects/db029da6fcef496ba63018aa203aa3c0/users/14c47af8f963444f93307b6c6bd00a52/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:13.730480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141272f-3949-446f-a514-15a3d7dc8f7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=db029da6fcef496ba63018aa203aa3c0, user_id=14c47af8f963444f93307b6c6bd00a52, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:13.733292 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141272f-3949-446f-a514-15a3d7dc8f7d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:13.757061 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1296/2604] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:13 2022] PUT /identity/v3/projects/db029da6fcef496ba63018aa203aa3c0/users/14c47af8f963444f93307b6c6bd00a52/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:13.759145 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf1d4f-e4ab-4d56-982c-5f2d3870c941 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:13.759441 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf1d4f-e4ab-4d56-982c-5f2d3870c941 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:13.759721 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf1d4f-e4ab-4d56-982c-5f2d3870c941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:13.767911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:13.795369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-adcf1d4f-e4ab-4d56-982c-5f2d3870c941 None None] MFA Rules not processed for user `14c47af8f963444f93307b6c6bd00a52`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:13.797669 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-adcf1d4f-e4ab-4d56-982c-5f2d3870c941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:13.906947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1309/2605] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:07:13.917476 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141272f-3949-446f-a514-15a3d7dc8f7d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:13.919770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d141272f-3949-446f-a514-15a3d7dc8f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:14.036747 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141272f-3949-446f-a514-15a3d7dc8f7d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:14.049857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcb5d0-2660-46fc-af12-b0a708a728fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:14.050507 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcb5d0-2660-46fc-af12-b0a708a728fd service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:14.050806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcb5d0-2660-46fc-af12-b0a708a728fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:14.051086 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcb5d0-2660-46fc-af12-b0a708a728fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:14.068135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcb5d0-2660-46fc-af12-b0a708a728fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:14.069325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcb5d0-2660-46fc-af12-b0a708a728fd service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:14.093107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1297/2606] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:14.370983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff864c64-af9c-4a22-b6f3-70734ffdc4a8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:14.371472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff864c64-af9c-4a22-b6f3-70734ffdc4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:14.371569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff864c64-af9c-4a22-b6f3-70734ffdc4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:14.403735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ff864c64-af9c-4a22-b6f3-70734ffdc4a8 None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:14.406993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ff864c64-af9c-4a22-b6f3-70734ffdc4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:14.504423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1310/2607] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 04:07:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:14.519054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcb5d0-2660-46fc-af12-b0a708a728fd service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:14.521436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a7bcb5d0-2660-46fc-af12-b0a708a728fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:14.544956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:14.653363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcb5d0-2660-46fc-af12-b0a708a728fd service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:14.681909 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a900cbd-4dd2-4a32-8c36-0cd07129e46f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:14.690999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a900cbd-4dd2-4a32-8c36-0cd07129e46f service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:14.691527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a900cbd-4dd2-4a32-8c36-0cd07129e46f service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:14.691884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a900cbd-4dd2-4a32-8c36-0cd07129e46f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:14.703930 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a900cbd-4dd2-4a32-8c36-0cd07129e46f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:14.705990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a900cbd-4dd2-4a32-8c36-0cd07129e46f service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:14.723330 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1298/2608] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:07:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:07:15.003433 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff864c64-af9c-4a22-b6f3-70734ffdc4a8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:15.004962 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ff864c64-af9c-4a22-b6f3-70734ffdc4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:15.028369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff864c64-af9c-4a22-b6f3-70734ffdc4a8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:15.037889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c17274-f085-47df-8396-1f1a4aabecc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:15.038756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c17274-f085-47df-8396-1f1a4aabecc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:15.039100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c17274-f085-47df-8396-1f1a4aabecc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:15.039527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c17274-f085-47df-8396-1f1a4aabecc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:15.134174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b75e21-acba-4a52-9e6f-67548462be6f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:15.134603 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b75e21-acba-4a52-9e6f-67548462be6f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:15.140302 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b75e21-acba-4a52-9e6f-67548462be6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:15.234143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c17274-f085-47df-8396-1f1a4aabecc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:15.237044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c17274-f085-47df-8396-1f1a4aabecc6 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:15.247504 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-77b75e21-acba-4a52-9e6f-67548462be6f None None] MFA Rules not processed for user `c992bd261bdb418cb1a4b604a8e1bd8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:15.250596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-77b75e21-acba-4a52-9e6f-67548462be6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:15.268934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1311/2609] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:15.369576 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1299/2610] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:15.377498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c17274-f085-47df-8396-1f1a4aabecc6 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:15.379731 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-70c17274-f085-47df-8396-1f1a4aabecc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:15.440986 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:15.486028 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c17274-f085-47df-8396-1f1a4aabecc6 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:15.518599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25615a7-3262-44b4-8633-c05de084894d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8967641dd624dacb5ff12c8e22599bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:15.519342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25615a7-3262-44b4-8633-c05de084894d service glance] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:15.519801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25615a7-3262-44b4-8633-c05de084894d service glance] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:15.520092 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25615a7-3262-44b4-8633-c05de084894d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:15.523678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e25615a7-3262-44b4-8633-c05de084894d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:15.687393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25615a7-3262-44b4-8633-c05de084894d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:15.691792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25615a7-3262-44b4-8633-c05de084894d service glance] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:15.725557 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1312/2611] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:16.040983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97294034-9ea3-42a7-84c4-45b02be6b6e0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:16.041540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97294034-9ea3-42a7-84c4-45b02be6b6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:16.041878 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97294034-9ea3-42a7-84c4-45b02be6b6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:16.083808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-97294034-9ea3-42a7-84c4-45b02be6b6e0 None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:16.085100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-97294034-9ea3-42a7-84c4-45b02be6b6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:16.094561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1300/2612] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 04:07:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:16.117474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25615a7-3262-44b4-8633-c05de084894d service glance] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:16.132369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25615a7-3262-44b4-8633-c05de084894d service glance] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:16.141529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c5704-02ed-42da-b864-a4852aa11a94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:16.142641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c5704-02ed-42da-b864-a4852aa11a94 service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:16.143365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c5704-02ed-42da-b864-a4852aa11a94 service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:16.144244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c5704-02ed-42da-b864-a4852aa11a94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:16.163136 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c5704-02ed-42da-b864-a4852aa11a94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:16.164802 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c5704-02ed-42da-b864-a4852aa11a94 service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:16.190327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1313/2613] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:07:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:07:20.134118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97294034-9ea3-42a7-84c4-45b02be6b6e0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:20.149186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97294034-9ea3-42a7-84c4-45b02be6b6e0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:20.153701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a905b-a130-4d88-b9b9-884e6e254559 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:20.155652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a905b-a130-4d88-b9b9-884e6e254559 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:20.155968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a905b-a130-4d88-b9b9-884e6e254559 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:20.156412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a905b-a130-4d88-b9b9-884e6e254559 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:20.255959 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a905b-a130-4d88-b9b9-884e6e254559 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:20.257460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a905b-a130-4d88-b9b9-884e6e254559 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:20.273031 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1301/2614] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:23.429558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c5704-02ed-42da-b864-a4852aa11a94 service swift] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:23.594665 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c5704-02ed-42da-b864-a4852aa11a94 service swift] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:23.600433 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:23.601141 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:23.601471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:23.601757 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] PATH_INFO: `/v3/users/f33a0683b1dd44cd9164f5636f8f1006` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:23.602997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f33a0683b1dd44cd9164f5636f8f1006)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:23.605002 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:23.671715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f33a0683b1dd44cd9164f5636f8f1006'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:23.673059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f33a0683b1dd44cd9164f5636f8f1006'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:23.690596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f33a0683b1dd44cd9164f5636f8f1006'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:23.699018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f33a0683b1dd44cd9164f5636f8f1006'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:23.701132 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:23.790270 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f33a0683b1dd44cd9164f5636f8f1006'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:23.809573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1314/2615] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:23 2022] DELETE /identity/v3/users/f33a0683b1dd44cd9164f5636f8f1006 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:23.815093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1627c-b120-46c2-abd1-f65da809a171 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:23.815990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1627c-b120-46c2-abd1-f65da809a171 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:23.816583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1627c-b120-46c2-abd1-f65da809a171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:23.847311 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-26d1627c-b120-46c2-abd1-f65da809a171 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:23.849366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-26d1627c-b120-46c2-abd1-f65da809a171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:23.861172 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:23.945395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1302/2616] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:23.966042 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:23.968516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:24.307044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa38ee-aece-4d98-93ef-fb751251a77b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:24.319126 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708f8dd-8c54-474f-8676-f428c63cb2ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:24.320197 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708f8dd-8c54-474f-8676-f428c63cb2ef service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:24.322027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708f8dd-8c54-474f-8676-f428c63cb2ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:24.322464 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708f8dd-8c54-474f-8676-f428c63cb2ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:24.344707 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708f8dd-8c54-474f-8676-f428c63cb2ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:24.347024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708f8dd-8c54-474f-8676-f428c63cb2ef service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:24.370633 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1315/2617] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:24.870113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1627c-b120-46c2-abd1-f65da809a171 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:24.873991 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-26d1627c-b120-46c2-abd1-f65da809a171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:24.928263 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1627c-b120-46c2-abd1-f65da809a171 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:24.939510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:24.940428 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:24.940877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:24.941317 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] PATH_INFO: `/v3/projects/855a939a8fe8445d9bb11f6959451f0a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:24.944125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=855a939a8fe8445d9bb11f6959451f0a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:24.946606 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:25.008781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '855a939a8fe8445d9bb11f6959451f0a', 'request_id': 'req-c6b34bdc-2714-4406-8eca-b96f33f0f608'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:25.016495 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '855a939a8fe8445d9bb11f6959451f0a', 'request_id': 'req-c6b34bdc-2714-4406-8eca-b96f33f0f608'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:25.025758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '855a939a8fe8445d9bb11f6959451f0a', 'request_id': 'req-c6b34bdc-2714-4406-8eca-b96f33f0f608'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:25.039537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] The token cache is being invalidate because project 855a939a8fe8445d9bb11f6959451f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:25.039884 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:25.040506 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:25.042253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1303/2618] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:24 2022] DELETE /identity/v3/projects/855a939a8fe8445d9bb11f6959451f0a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:25.046192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708f8dd-8c54-474f-8676-f428c63cb2ef service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:25.056918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708f8dd-8c54-474f-8676-f428c63cb2ef service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:25.120181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:25.121098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:25.121550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:25.121980 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] PATH_INFO: `/v3/users/e70f77bdf6634aa3b3de5b075af6f587` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:25.123379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e70f77bdf6634aa3b3de5b075af6f587)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:25.125527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:25.181908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e70f77bdf6634aa3b3de5b075af6f587'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:25.183157 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e70f77bdf6634aa3b3de5b075af6f587'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:25.198573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e70f77bdf6634aa3b3de5b075af6f587'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:25.206498 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e70f77bdf6634aa3b3de5b075af6f587'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:25.215212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e70f77bdf6634aa3b3de5b075af6f587'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:25.226824 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1316/2619] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:25 2022] DELETE /identity/v3/users/e70f77bdf6634aa3b3de5b075af6f587 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:25.829529 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:25.832577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:25.849468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b34bdc-2714-4406-8eca-b96f33f0f608 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:25.952714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:25.955126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:25.955759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:25.957377 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] PATH_INFO: `/v3/projects/ca88928b7ccc446a8380804c4860b113` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:25.963006 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca88928b7ccc446a8380804c4860b113)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:25.971347 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:26.038545 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:26.051312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca88928b7ccc446a8380804c4860b113', 'request_id': 'req-65eb08fc-555f-48c3-8e62-ce217c5a0535'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:26.060662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca88928b7ccc446a8380804c4860b113', 'request_id': 'req-65eb08fc-555f-48c3-8e62-ce217c5a0535'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:26.076450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca88928b7ccc446a8380804c4860b113', 'request_id': 'req-65eb08fc-555f-48c3-8e62-ce217c5a0535'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:26.098187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] The token cache is being invalidate because project ca88928b7ccc446a8380804c4860b113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:26.098593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:26.099650 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:26.101984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1304/2620] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:25 2022] DELETE /identity/v3/projects/ca88928b7ccc446a8380804c4860b113 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:26.106536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:26.108614 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:26.126094 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485bf0a-261c-4f1f-a209-e00aafc15e2e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:26.137597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:26.237833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58325f39-6ac7-45d5-bff0-b2744dac0a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:26.238518 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58325f39-6ac7-45d5-bff0-b2744dac0a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:26.238977 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58325f39-6ac7-45d5-bff0-b2744dac0a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:26.239104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58325f39-6ac7-45d5-bff0-b2744dac0a9d admin admin] PATH_INFO: `/v3/users/14c47af8f963444f93307b6c6bd00a52` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:26.240287 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58325f39-6ac7-45d5-bff0-b2744dac0a9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c47af8f963444f93307b6c6bd00a52)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:26.242283 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58325f39-6ac7-45d5-bff0-b2744dac0a9d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:26.347385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-58325f39-6ac7-45d5-bff0-b2744dac0a9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c47af8f963444f93307b6c6bd00a52'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:26.347385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-58325f39-6ac7-45d5-bff0-b2744dac0a9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c47af8f963444f93307b6c6bd00a52'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:26.372109 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-58325f39-6ac7-45d5-bff0-b2744dac0a9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c47af8f963444f93307b6c6bd00a52'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:26.390891 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-58325f39-6ac7-45d5-bff0-b2744dac0a9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c47af8f963444f93307b6c6bd00a52'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:26.399590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-58325f39-6ac7-45d5-bff0-b2744dac0a9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c47af8f963444f93307b6c6bd00a52'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:26.420998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1317/2621] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:26 2022] DELETE /identity/v3/users/14c47af8f963444f93307b6c6bd00a52 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:26.963147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:26.965477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:27.021160 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb08fc-555f-48c3-8e62-ce217c5a0535 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:27.125131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:27.126153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:27.126647 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:27.127199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] PATH_INFO: `/v3/projects/db029da6fcef496ba63018aa203aa3c0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:27.128639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db029da6fcef496ba63018aa203aa3c0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:27.131429 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:27.242437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db029da6fcef496ba63018aa203aa3c0', 'request_id': 'req-42892d16-17a4-46e0-8c18-96d7273bbde4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:27.249108 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db029da6fcef496ba63018aa203aa3c0', 'request_id': 'req-42892d16-17a4-46e0-8c18-96d7273bbde4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:27.263413 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db029da6fcef496ba63018aa203aa3c0', 'request_id': 'req-42892d16-17a4-46e0-8c18-96d7273bbde4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:27.281466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] The token cache is being invalidate because project db029da6fcef496ba63018aa203aa3c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:27.282121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:27.283162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-42892d16-17a4-46e0-8c18-96d7273bbde4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:27.285282 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1305/2622] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:26 2022] DELETE /identity/v3/projects/db029da6fcef496ba63018aa203aa3c0 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:07:27.293213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7d922-98d4-46fa-a5e7-4c0fa27498b8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:27.293843 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7d922-98d4-46fa-a5e7-4c0fa27498b8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:27.293843 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7d922-98d4-46fa-a5e7-4c0fa27498b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:27.326927 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-00a7d922-98d4-46fa-a5e7-4c0fa27498b8 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:27.329550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-00a7d922-98d4-46fa-a5e7-4c0fa27498b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:27.446953 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1318/2623] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:27.529450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb5538-2110-4cf0-9dee-8b2348fc29d2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:27.529735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb5538-2110-4cf0-9dee-8b2348fc29d2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:27.530011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb5538-2110-4cf0-9dee-8b2348fc29d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:27.568065 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-60bb5538-2110-4cf0-9dee-8b2348fc29d2 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:27.569361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-60bb5538-2110-4cf0-9dee-8b2348fc29d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:27.582744 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1306/2624] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:27.584321 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7d922-98d4-46fa-a5e7-4c0fa27498b8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:27.596745 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7d922-98d4-46fa-a5e7-4c0fa27498b8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:27.601588 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9831946-5ce5-47dc-a1c3-bafdb316af8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:27.602281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9831946-5ce5-47dc-a1c3-bafdb316af8b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:27.602638 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9831946-5ce5-47dc-a1c3-bafdb316af8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:27.606857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9831946-5ce5-47dc-a1c3-bafdb316af8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:27.606857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9831946-5ce5-47dc-a1c3-bafdb316af8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:07:27.606857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9831946-5ce5-47dc-a1c3-bafdb316af8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:27.606857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9831946-5ce5-47dc-a1c3-bafdb316af8b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:27.627878 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1319/2625] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:07:27.632287 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb5538-2110-4cf0-9dee-8b2348fc29d2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:27.642351 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb5538-2110-4cf0-9dee-8b2348fc29d2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:27.647102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f0d8b-759a-42f9-a362-985b5d13d0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:27.647844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f0d8b-759a-42f9-a362-985b5d13d0cf admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:27.648192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f0d8b-759a-42f9-a362-985b5d13d0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:27.648533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f0d8b-759a-42f9-a362-985b5d13d0cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:27.649270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f0d8b-759a-42f9-a362-985b5d13d0cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:27.651307 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f0d8b-759a-42f9-a362-985b5d13d0cf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:27.685664 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1307/2626] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:27 2022] POST /identity/v3/projects => generated 401 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:27.690652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9831946-5ce5-47dc-a1c3-bafdb316af8b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:27.701025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9831946-5ce5-47dc-a1c3-bafdb316af8b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:27.721113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:27.769144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6ed60-59ed-45a6-8e1b-353d211d613d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:27.769873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6ed60-59ed-45a6-8e1b-353d211d613d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:27.770225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6ed60-59ed-45a6-8e1b-353d211d613d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:27.770589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6ed60-59ed-45a6-8e1b-353d211d613d admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:27.771357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6ed60-59ed-45a6-8e1b-353d211d613d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:27.773325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6ed60-59ed-45a6-8e1b-353d211d613d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:27.830257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1320/2627] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:27 2022] POST /identity/v3/users => generated 301 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:27.831688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f0d8b-759a-42f9-a362-985b5d13d0cf admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:27.880075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:27.905168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f0d8b-759a-42f9-a362-985b5d13d0cf admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:27.911330 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364eed4-1720-4381-9629-61c509ab4c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:27.912328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364eed4-1720-4381-9629-61c509ab4c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:27.912777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364eed4-1720-4381-9629-61c509ab4c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:27.913208 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364eed4-1720-4381-9629-61c509ab4c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:27.914095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364eed4-1720-4381-9629-61c509ab4c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:27.915768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364eed4-1720-4381-9629-61c509ab4c3b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:27.926817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1308/2628] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:27.930673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6ed60-59ed-45a6-8e1b-353d211d613d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:27.986599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6ed60-59ed-45a6-8e1b-353d211d613d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:27.991511 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c0d97-a661-4803-84b8-642da6077bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:27.992123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0d97-a661-4803-84b8-642da6077bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:27.992465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0d97-a661-4803-84b8-642da6077bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:27.992751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0d97-a661-4803-84b8-642da6077bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:27.993448 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c0d97-a661-4803-84b8-642da6077bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:27.995086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c0d97-a661-4803-84b8-642da6077bcd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:28.006925 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1321/2629] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:07:28.010947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364eed4-1720-4381-9629-61c509ab4c3b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:28.021754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364eed4-1720-4381-9629-61c509ab4c3b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:28.026627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8718861-b215-4ec2-8992-97190d6218ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:28.027420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8718861-b215-4ec2-8992-97190d6218ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.027776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8718861-b215-4ec2-8992-97190d6218ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.028122 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8718861-b215-4ec2-8992-97190d6218ba admin admin] PATH_INFO: `/v3/projects/7f7171d932bc4c2ab88f16f14bdf80a0/users/b0c8659545d44932ada449e125c21e6b/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:28.047489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8718861-b215-4ec2-8992-97190d6218ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7171d932bc4c2ab88f16f14bdf80a0, user_id=b0c8659545d44932ada449e125c21e6b, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:28.050290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8718861-b215-4ec2-8992-97190d6218ba admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:28.065761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1309/2630] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:28 2022] PUT /identity/v3/projects/7f7171d932bc4c2ab88f16f14bdf80a0/users/b0c8659545d44932ada449e125c21e6b/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:07:28.077296 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0837ee4-c876-4ac3-9b9a-4d9940619486 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.078312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0837ee4-c876-4ac3-9b9a-4d9940619486 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.078596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0837ee4-c876-4ac3-9b9a-4d9940619486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:28.108246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d0837ee4-c876-4ac3-9b9a-4d9940619486 None None] MFA Rules not processed for user `b0c8659545d44932ada449e125c21e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:28.109516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d0837ee4-c876-4ac3-9b9a-4d9940619486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:28.182372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1322/2631] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:28.187384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8718861-b215-4ec2-8992-97190d6218ba admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:28.258022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8718861-b215-4ec2-8992-97190d6218ba admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:28.263854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e593850-7925-414b-8fc6-b6e166dcaef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:28.264477 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593850-7925-414b-8fc6-b6e166dcaef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.264757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593850-7925-414b-8fc6-b6e166dcaef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.265394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593850-7925-414b-8fc6-b6e166dcaef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:28.266049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e593850-7925-414b-8fc6-b6e166dcaef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:28.268170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e593850-7925-414b-8fc6-b6e166dcaef8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:28.268478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38df8e-ee22-4004-9858-d3326995ee25 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.268788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38df8e-ee22-4004-9858-d3326995ee25 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.269052 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38df8e-ee22-4004-9858-d3326995ee25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:28.295356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bb38df8e-ee22-4004-9858-d3326995ee25 None None] MFA Rules not processed for user `9d30603fefe44a0c9dbed8c170071b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:28.297310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bb38df8e-ee22-4004-9858-d3326995ee25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:28.317191 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1310/2632] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:28 2022] POST /identity/v3/projects => generated 399 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:28.319099 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e593850-7925-414b-8fc6-b6e166dcaef8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:28.324391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:28.370909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1323/2633] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:07:28.394722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e593850-7925-414b-8fc6-b6e166dcaef8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:28.399887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab927ba-01ec-46d4-a38d-463bf4b1630b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:28.400510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab927ba-01ec-46d4-a38d-463bf4b1630b admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.400797 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab927ba-01ec-46d4-a38d-463bf4b1630b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.401076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab927ba-01ec-46d4-a38d-463bf4b1630b admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:28.401811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab927ba-01ec-46d4-a38d-463bf4b1630b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:28.403848 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab927ba-01ec-46d4-a38d-463bf4b1630b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:28.430282 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1311/2634] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:28 2022] POST /identity/v3/users => generated 306 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:28.431451 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38df8e-ee22-4004-9858-d3326995ee25 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:28.442621 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38df8e-ee22-4004-9858-d3326995ee25 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:28.447296 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db4b4f-1f16-4a94-ab0f-7477abb3f13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:28.447915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db4b4f-1f16-4a94-ab0f-7477abb3f13f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.448199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db4b4f-1f16-4a94-ab0f-7477abb3f13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.448472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db4b4f-1f16-4a94-ab0f-7477abb3f13f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:28.449200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db4b4f-1f16-4a94-ab0f-7477abb3f13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:28.450793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db4b4f-1f16-4a94-ab0f-7477abb3f13f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:28.461976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1324/2635] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:07:28.466235 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab927ba-01ec-46d4-a38d-463bf4b1630b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:28.477379 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab927ba-01ec-46d4-a38d-463bf4b1630b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:28.482262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dade964-676e-4ee7-9799-2c232832e1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:28.482967 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dade964-676e-4ee7-9799-2c232832e1a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.483258 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dade964-676e-4ee7-9799-2c232832e1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.483541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dade964-676e-4ee7-9799-2c232832e1a4 admin admin] PATH_INFO: `/v3/projects/b9b1ffd3d5f44f25ae2d42993d2ff352/users/c44530c7cae14309b62124471dfe630b/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:28.500219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dade964-676e-4ee7-9799-2c232832e1a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b1ffd3d5f44f25ae2d42993d2ff352, user_id=c44530c7cae14309b62124471dfe630b, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:28.502941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dade964-676e-4ee7-9799-2c232832e1a4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:28.516784 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1312/2636] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:28 2022] PUT /identity/v3/projects/b9b1ffd3d5f44f25ae2d42993d2ff352/users/c44530c7cae14309b62124471dfe630b/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:28.520946 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db4b4f-1f16-4a94-ab0f-7477abb3f13f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:28.530952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db4b4f-1f16-4a94-ab0f-7477abb3f13f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:28.592357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cbe76-6a23-4b20-9856-0e23aedea89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:28.593787 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cbe76-6a23-4b20-9856-0e23aedea89e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.594206 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cbe76-6a23-4b20-9856-0e23aedea89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.594592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cbe76-6a23-4b20-9856-0e23aedea89e admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:28.595455 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cbe76-6a23-4b20-9856-0e23aedea89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:28.597075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cbe76-6a23-4b20-9856-0e23aedea89e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:28.607490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1325/2637] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:07:28.614563 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dade964-676e-4ee7-9799-2c232832e1a4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:28.621231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:28.625356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dade964-676e-4ee7-9799-2c232832e1a4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:28.631237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaec7b5-f925-4fe6-b261-c54527639ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:28.632359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaec7b5-f925-4fe6-b261-c54527639ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.632726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaec7b5-f925-4fe6-b261-c54527639ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.633012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaec7b5-f925-4fe6-b261-c54527639ae7 admin admin] PATH_INFO: `/v3/projects/b9b1ffd3d5f44f25ae2d42993d2ff352/users/c44530c7cae14309b62124471dfe630b/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:28.635140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaec7b5-f925-4fe6-b261-c54527639ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b1ffd3d5f44f25ae2d42993d2ff352, user_id=c44530c7cae14309b62124471dfe630b, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:28.637854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaec7b5-f925-4fe6-b261-c54527639ae7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:28.649394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1313/2638] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:28 2022] PUT /identity/v3/projects/b9b1ffd3d5f44f25ae2d42993d2ff352/users/c44530c7cae14309b62124471dfe630b/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:28.662310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08718fb-40b1-4c42-9ca4-f6d8f80bdbe6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.662628 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08718fb-40b1-4c42-9ca4-f6d8f80bdbe6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.662942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08718fb-40b1-4c42-9ca4-f6d8f80bdbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:28.691101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e08718fb-40b1-4c42-9ca4-f6d8f80bdbe6 None None] MFA Rules not processed for user `c44530c7cae14309b62124471dfe630b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:28.692312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e08718fb-40b1-4c42-9ca4-f6d8f80bdbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:28.763385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1326/2639] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:28 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:28.775041 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaec7b5-f925-4fe6-b261-c54527639ae7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:28.776828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9eaec7b5-f925-4fe6-b261-c54527639ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:28.864701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaec7b5-f925-4fe6-b261-c54527639ae7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:28.868409 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b86eee-9f46-4a48-ac6e-cbbd2285c6a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:28.869157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b86eee-9f46-4a48-ac6e-cbbd2285c6a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.869472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b86eee-9f46-4a48-ac6e-cbbd2285c6a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.869754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b86eee-9f46-4a48-ac6e-cbbd2285c6a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:28.879212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b86eee-9f46-4a48-ac6e-cbbd2285c6a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:28.880697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b86eee-9f46-4a48-ac6e-cbbd2285c6a4 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:28.890844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1314/2640] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:28 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:28.975138 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08718fb-40b1-4c42-9ca4-f6d8f80bdbe6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:28.987438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08718fb-40b1-4c42-9ca4-f6d8f80bdbe6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:28.991429 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548354ba-935b-474b-81c5-0c119edbb640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:28.992452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548354ba-935b-474b-81c5-0c119edbb640 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:28.992736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548354ba-935b-474b-81c5-0c119edbb640 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:28.993268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548354ba-935b-474b-81c5-0c119edbb640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:29.043704 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:29.060268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548354ba-935b-474b-81c5-0c119edbb640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:29.061623 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548354ba-935b-474b-81c5-0c119edbb640 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:29.071218 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1327/2641] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:28 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:30.140641 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f435c-04a6-4a81-8e38-5cc022b076d5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:30.141123 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f435c-04a6-4a81-8e38-5cc022b076d5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:30.141240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f435c-04a6-4a81-8e38-5cc022b076d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:30.170758 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4e5f435c-04a6-4a81-8e38-5cc022b076d5 None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:30.172637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4e5f435c-04a6-4a81-8e38-5cc022b076d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:30.239156 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1315/2642] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 04:07:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:30.252983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548354ba-935b-474b-81c5-0c119edbb640 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:30.254476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-548354ba-935b-474b-81c5-0c119edbb640 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:30.325667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548354ba-935b-474b-81c5-0c119edbb640 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:30.331419 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be3b46-0318-4dac-aacb-24210199d98f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:30.332158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be3b46-0318-4dac-aacb-24210199d98f service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:30.332443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be3b46-0318-4dac-aacb-24210199d98f service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:30.332715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be3b46-0318-4dac-aacb-24210199d98f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:30.340937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be3b46-0318-4dac-aacb-24210199d98f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:30.342521 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be3b46-0318-4dac-aacb-24210199d98f service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:30.350751 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1328/2643] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:07:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:07:33.807386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f435c-04a6-4a81-8e38-5cc022b076d5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:33.915225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f435c-04a6-4a81-8e38-5cc022b076d5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:33.923633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:33.925202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:33.925842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:33.926403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] PATH_INFO: `/v3/users/b0c8659545d44932ada449e125c21e6b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:33.927995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0c8659545d44932ada449e125c21e6b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:33.930631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:33.944881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:33.993374 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0c8659545d44932ada449e125c21e6b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:33.994034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0c8659545d44932ada449e125c21e6b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.002140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0c8659545d44932ada449e125c21e6b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.013958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0c8659545d44932ada449e125c21e6b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.025751 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0c8659545d44932ada449e125c21e6b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.046029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1316/2644] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:33 2022] DELETE /identity/v3/users/b0c8659545d44932ada449e125c21e6b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:34.050339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718f2c0-7249-4ff1-aae3-3f846622b927 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:34.050905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718f2c0-7249-4ff1-aae3-3f846622b927 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:34.051011 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718f2c0-7249-4ff1-aae3-3f846622b927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:34.086567 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7718f2c0-7249-4ff1-aae3-3f846622b927 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:34.088183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7718f2c0-7249-4ff1-aae3-3f846622b927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:34.151673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1329/2645] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:34.162246 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:34.163725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:34.232918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4a2e5-e1a0-42a7-b395-072eae0d4ec7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:34.237532 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218955dc-0682-4b5d-8496-4ee2628765d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:34.238167 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218955dc-0682-4b5d-8496-4ee2628765d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:34.238452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218955dc-0682-4b5d-8496-4ee2628765d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:34.238727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218955dc-0682-4b5d-8496-4ee2628765d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:34.249688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218955dc-0682-4b5d-8496-4ee2628765d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:34.251227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218955dc-0682-4b5d-8496-4ee2628765d5 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:34.273652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1317/2646] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:34.606458 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718f2c0-7249-4ff1-aae3-3f846622b927 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:34.607976 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7718f2c0-7249-4ff1-aae3-3f846622b927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:34.623036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718f2c0-7249-4ff1-aae3-3f846622b927 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:34.630392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:34.631539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:34.632089 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:34.632452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] PATH_INFO: `/v3/projects/7f7171d932bc4c2ab88f16f14bdf80a0` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:34.634303 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f7171d932bc4c2ab88f16f14bdf80a0)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:34.636892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:34.697594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f7171d932bc4c2ab88f16f14bdf80a0', 'request_id': 'req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.704577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f7171d932bc4c2ab88f16f14bdf80a0', 'request_id': 'req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.707578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:34.714022 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f7171d932bc4c2ab88f16f14bdf80a0', 'request_id': 'req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.732930 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] The token cache is being invalidate because project 7f7171d932bc4c2ab88f16f14bdf80a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:34.733354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.734202 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.736191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1330/2647] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:34 2022] DELETE /identity/v3/projects/7f7171d932bc4c2ab88f16f14bdf80a0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:07:34.740089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218955dc-0682-4b5d-8496-4ee2628765d5 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:34.751310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218955dc-0682-4b5d-8496-4ee2628765d5 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:34.856249 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b95ba9-2982-4a57-ba78-a0af678f9685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:34.857019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b95ba9-2982-4a57-ba78-a0af678f9685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:34.857119 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b95ba9-2982-4a57-ba78-a0af678f9685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:34.857388 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b95ba9-2982-4a57-ba78-a0af678f9685 admin admin] PATH_INFO: `/v3/users/c44530c7cae14309b62124471dfe630b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:34.859267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b95ba9-2982-4a57-ba78-a0af678f9685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c44530c7cae14309b62124471dfe630b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:34.861245 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b95ba9-2982-4a57-ba78-a0af678f9685 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:34.909030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:34.924525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b8b95ba9-2982-4a57-ba78-a0af678f9685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c44530c7cae14309b62124471dfe630b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.925136 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b8b95ba9-2982-4a57-ba78-a0af678f9685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c44530c7cae14309b62124471dfe630b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.932600 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b8b95ba9-2982-4a57-ba78-a0af678f9685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c44530c7cae14309b62124471dfe630b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.945875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b8b95ba9-2982-4a57-ba78-a0af678f9685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c44530c7cae14309b62124471dfe630b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.961496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b8b95ba9-2982-4a57-ba78-a0af678f9685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c44530c7cae14309b62124471dfe630b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:34.981219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1318/2648] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:34 2022] DELETE /identity/v3/users/c44530c7cae14309b62124471dfe630b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:35.363781 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:35.366358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:35.382678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c36ea-ebaa-4f1c-a09f-152b2cbbc2ac admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:35.465020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:35.465680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:35.465961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:35.466240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] PATH_INFO: `/v3/projects/b9b1ffd3d5f44f25ae2d42993d2ff352` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:35.467444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9b1ffd3d5f44f25ae2d42993d2ff352)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:35.469928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:35.530367 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9b1ffd3d5f44f25ae2d42993d2ff352', 'request_id': 'req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:35.545005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9b1ffd3d5f44f25ae2d42993d2ff352', 'request_id': 'req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:35.559700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9b1ffd3d5f44f25ae2d42993d2ff352', 'request_id': 'req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:35.578890 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] The token cache is being invalidate because project b9b1ffd3d5f44f25ae2d42993d2ff352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:35.579450 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:35.580421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bb244a0f-7abc-4a0d-b39a-5a72096f7b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:35.582165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1331/2649] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:35 2022] DELETE /identity/v3/projects/b9b1ffd3d5f44f25ae2d42993d2ff352 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:35.587224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fed26-720d-4d53-94c7-4c2524e34082 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:35.587853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fed26-720d-4d53-94c7-4c2524e34082 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:35.588413 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fed26-720d-4d53-94c7-4c2524e34082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:35.666586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-328fed26-720d-4d53-94c7-4c2524e34082 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:35.667970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-328fed26-720d-4d53-94c7-4c2524e34082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:35.764796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1319/2650] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:35.773496 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55701738-e954-49d7-92fb-d77f9f37bcea None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:35.773846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55701738-e954-49d7-92fb-d77f9f37bcea None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:35.774078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55701738-e954-49d7-92fb-d77f9f37bcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:35.799993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-55701738-e954-49d7-92fb-d77f9f37bcea None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:35.801232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-55701738-e954-49d7-92fb-d77f9f37bcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:35.815201 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1332/2651] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:35.817017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fed26-720d-4d53-94c7-4c2524e34082 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:35.829124 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fed26-720d-4d53-94c7-4c2524e34082 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:35.834291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217726f-7c62-491c-8846-3be3d332d570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:35.835190 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217726f-7c62-491c-8846-3be3d332d570 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:35.835614 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217726f-7c62-491c-8846-3be3d332d570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:35.835919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217726f-7c62-491c-8846-3be3d332d570 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:35.837046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217726f-7c62-491c-8846-3be3d332d570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:07:35.837363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217726f-7c62-491c-8846-3be3d332d570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:35.839076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217726f-7c62-491c-8846-3be3d332d570 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:35.851730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1320/2652] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:07:35.856268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55701738-e954-49d7-92fb-d77f9f37bcea None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:35.866378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55701738-e954-49d7-92fb-d77f9f37bcea None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:35.871108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a56c03-3918-4e46-ad79-db0f4b3e024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:35.871714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a56c03-3918-4e46-ad79-db0f4b3e024d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:35.872000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a56c03-3918-4e46-ad79-db0f4b3e024d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:35.872279 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a56c03-3918-4e46-ad79-db0f4b3e024d admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:35.872943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a56c03-3918-4e46-ad79-db0f4b3e024d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:35.875514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a56c03-3918-4e46-ad79-db0f4b3e024d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:35.901949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1333/2653] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:35 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:35.906406 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217726f-7c62-491c-8846-3be3d332d570 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:35.918869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217726f-7c62-491c-8846-3be3d332d570 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:35.974981 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f111912-f898-4ced-aab0-63ff397fd36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:35.975597 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f111912-f898-4ced-aab0-63ff397fd36d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:35.975885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f111912-f898-4ced-aab0-63ff397fd36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:35.976165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f111912-f898-4ced-aab0-63ff397fd36d admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:35.976832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f111912-f898-4ced-aab0-63ff397fd36d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:35.978757 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f111912-f898-4ced-aab0-63ff397fd36d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:36.000740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1321/2654] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:35 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:36.004874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a56c03-3918-4e46-ad79-db0f4b3e024d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:36.016246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a56c03-3918-4e46-ad79-db0f4b3e024d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:36.024488 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8c222-3290-4248-9a3c-ce730cdad489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:36.025597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c222-3290-4248-9a3c-ce730cdad489 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:36.026269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c222-3290-4248-9a3c-ce730cdad489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:36.026938 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c222-3290-4248-9a3c-ce730cdad489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:36.028370 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8c222-3290-4248-9a3c-ce730cdad489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:36.031427 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8c222-3290-4248-9a3c-ce730cdad489 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:36.049960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1334/2655] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:36.054583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f111912-f898-4ced-aab0-63ff397fd36d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:36.065121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:36.078035 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f111912-f898-4ced-aab0-63ff397fd36d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:36.084309 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a04658-23c8-43c8-a2f9-b8c16e1aff75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:36.085110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a04658-23c8-43c8-a2f9-b8c16e1aff75 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:36.085399 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a04658-23c8-43c8-a2f9-b8c16e1aff75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:36.085854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a04658-23c8-43c8-a2f9-b8c16e1aff75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:36.086726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a04658-23c8-43c8-a2f9-b8c16e1aff75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:36.088663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a04658-23c8-43c8-a2f9-b8c16e1aff75 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:36.101568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1322/2656] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:07:36.105596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8c222-3290-4248-9a3c-ce730cdad489 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:36.116600 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8c222-3290-4248-9a3c-ce730cdad489 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:36.121963 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc739b1-57da-4672-be4e-134c0060c92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:36.122690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc739b1-57da-4672-be4e-134c0060c92b admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:36.123072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc739b1-57da-4672-be4e-134c0060c92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:36.123412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc739b1-57da-4672-be4e-134c0060c92b admin admin] PATH_INFO: `/v3/projects/70f0d1eb8f9b45a598a02c68d0e8818c/users/1c5dce796f24488a9cefcf16b8aae8b7/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:36.139709 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc739b1-57da-4672-be4e-134c0060c92b admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f0d1eb8f9b45a598a02c68d0e8818c, user_id=1c5dce796f24488a9cefcf16b8aae8b7, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:36.142353 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc739b1-57da-4672-be4e-134c0060c92b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:36.154821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1335/2657] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:36 2022] PUT /identity/v3/projects/70f0d1eb8f9b45a598a02c68d0e8818c/users/1c5dce796f24488a9cefcf16b8aae8b7/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:36.165401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba89701-a161-478c-88f2-2bcd62813abe None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:36.165715 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba89701-a161-478c-88f2-2bcd62813abe None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:36.165955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba89701-a161-478c-88f2-2bcd62813abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:36.195723 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-dba89701-a161-478c-88f2-2bcd62813abe None None] MFA Rules not processed for user `1c5dce796f24488a9cefcf16b8aae8b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:36.196924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dba89701-a161-478c-88f2-2bcd62813abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:36.261974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1323/2658] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:36.267113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc739b1-57da-4672-be4e-134c0060c92b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:36.285654 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:36.333234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc739b1-57da-4672-be4e-134c0060c92b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:36.337949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b4801-34d1-4231-aa0c-255284f8c3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:36.338573 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b4801-34d1-4231-aa0c-255284f8c3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:36.338900 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b4801-34d1-4231-aa0c-255284f8c3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:36.339181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b4801-34d1-4231-aa0c-255284f8c3d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:36.339844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b4801-34d1-4231-aa0c-255284f8c3d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:36.341742 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b4801-34d1-4231-aa0c-255284f8c3d1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:36.372315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1336/2659] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:36 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:07:36.378982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba89701-a161-478c-88f2-2bcd62813abe None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:36.394560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba89701-a161-478c-88f2-2bcd62813abe None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:36.465896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df773e0d-4edb-4e71-b89c-8d48784a4515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:36.466462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df773e0d-4edb-4e71-b89c-8d48784a4515 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:36.466766 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df773e0d-4edb-4e71-b89c-8d48784a4515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:36.467185 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df773e0d-4edb-4e71-b89c-8d48784a4515 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:36.468510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df773e0d-4edb-4e71-b89c-8d48784a4515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:36.471284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df773e0d-4edb-4e71-b89c-8d48784a4515 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:36.498346 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1324/2660] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:36 2022] POST /identity/v3/users => generated 301 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:36.499961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b4801-34d1-4231-aa0c-255284f8c3d1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:36.510729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b4801-34d1-4231-aa0c-255284f8c3d1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:36.515529 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadccf97-9b66-468b-aab7-dbe5fc0b8add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:36.516149 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadccf97-9b66-468b-aab7-dbe5fc0b8add admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:36.516434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadccf97-9b66-468b-aab7-dbe5fc0b8add admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:36.516711 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadccf97-9b66-468b-aab7-dbe5fc0b8add admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:36.517406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadccf97-9b66-468b-aab7-dbe5fc0b8add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:36.519041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadccf97-9b66-468b-aab7-dbe5fc0b8add admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:36.530832 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1337/2661] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:07:36.535242 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df773e0d-4edb-4e71-b89c-8d48784a4515 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:36.545682 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df773e0d-4edb-4e71-b89c-8d48784a4515 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:36.551116 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaf14f-1c6d-4aad-9f49-de88da125173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:36.551786 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaf14f-1c6d-4aad-9f49-de88da125173 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:36.552068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaf14f-1c6d-4aad-9f49-de88da125173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:36.552339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaf14f-1c6d-4aad-9f49-de88da125173 admin admin] PATH_INFO: `/v3/projects/c70ae97cbbcb4253b5acc62d7650f4e2/users/d41d491ae366427a8d5162ab503c7d7e/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:36.567237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaf14f-1c6d-4aad-9f49-de88da125173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c70ae97cbbcb4253b5acc62d7650f4e2, user_id=d41d491ae366427a8d5162ab503c7d7e, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:36.569873 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaf14f-1c6d-4aad-9f49-de88da125173 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:36.583587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1325/2662] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:36 2022] PUT /identity/v3/projects/c70ae97cbbcb4253b5acc62d7650f4e2/users/d41d491ae366427a8d5162ab503c7d7e/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:36.587613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadccf97-9b66-468b-aab7-dbe5fc0b8add admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:36.598346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadccf97-9b66-468b-aab7-dbe5fc0b8add admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:36.654312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f9f3c-a5f1-41d6-8892-283ad94cbc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:36.654965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f9f3c-a5f1-41d6-8892-283ad94cbc39 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:36.655825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f9f3c-a5f1-41d6-8892-283ad94cbc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:36.656121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f9f3c-a5f1-41d6-8892-283ad94cbc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:36.656818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f9f3c-a5f1-41d6-8892-283ad94cbc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:36.658410 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f9f3c-a5f1-41d6-8892-283ad94cbc39 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:36.668200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1338/2663] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:36.672460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaf14f-1c6d-4aad-9f49-de88da125173 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:36.682675 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaf14f-1c6d-4aad-9f49-de88da125173 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:36.687419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14cb0b-903d-4c7e-b61c-b2576c5e84b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:36.688069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14cb0b-903d-4c7e-b61c-b2576c5e84b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:36.688359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14cb0b-903d-4c7e-b61c-b2576c5e84b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:36.688641 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14cb0b-903d-4c7e-b61c-b2576c5e84b8 admin admin] PATH_INFO: `/v3/projects/c70ae97cbbcb4253b5acc62d7650f4e2/users/d41d491ae366427a8d5162ab503c7d7e/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:36.690561 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14cb0b-903d-4c7e-b61c-b2576c5e84b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c70ae97cbbcb4253b5acc62d7650f4e2, user_id=d41d491ae366427a8d5162ab503c7d7e, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:36.693237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14cb0b-903d-4c7e-b61c-b2576c5e84b8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:36.704020 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1326/2664] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:36 2022] PUT /identity/v3/projects/c70ae97cbbcb4253b5acc62d7650f4e2/users/d41d491ae366427a8d5162ab503c7d7e/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:36.715310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423c79f-3df5-4a74-8e7e-dbbfce0d0076 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:36.715603 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423c79f-3df5-4a74-8e7e-dbbfce0d0076 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:36.715881 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423c79f-3df5-4a74-8e7e-dbbfce0d0076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:36.758394 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9423c79f-3df5-4a74-8e7e-dbbfce0d0076 None None] MFA Rules not processed for user `d41d491ae366427a8d5162ab503c7d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:36.759619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9423c79f-3df5-4a74-8e7e-dbbfce0d0076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:36.857435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1339/2665] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:36.870632 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14cb0b-903d-4c7e-b61c-b2576c5e84b8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:36.872436 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5f14cb0b-903d-4c7e-b61c-b2576c5e84b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:36.910220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:37.063318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14cb0b-903d-4c7e-b61c-b2576c5e84b8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:37.067095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e622a2-c4c5-44a1-8423-babcac3c5ef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:37.067762 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e622a2-c4c5-44a1-8423-babcac3c5ef4 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:37.068045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e622a2-c4c5-44a1-8423-babcac3c5ef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:37.068336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e622a2-c4c5-44a1-8423-babcac3c5ef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:37.079075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e622a2-c4c5-44a1-8423-babcac3c5ef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:37.081007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e622a2-c4c5-44a1-8423-babcac3c5ef4 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:37.092825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1327/2666] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:37.244223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423c79f-3df5-4a74-8e7e-dbbfce0d0076 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:37.256867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:37.308889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423c79f-3df5-4a74-8e7e-dbbfce0d0076 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:37.314016 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:37.314640 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:37.314951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:37.315223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] PATH_INFO: `/v3/users/1c5dce796f24488a9cefcf16b8aae8b7` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:37.316363 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5dce796f24488a9cefcf16b8aae8b7)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:37.318289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:37.364546 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5dce796f24488a9cefcf16b8aae8b7'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:37.365213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5dce796f24488a9cefcf16b8aae8b7'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:37.379766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5dce796f24488a9cefcf16b8aae8b7'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:37.386708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5dce796f24488a9cefcf16b8aae8b7'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:37.394534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5dce796f24488a9cefcf16b8aae8b7'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:37.400609 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1340/2667] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:37 2022] DELETE /identity/v3/users/1c5dce796f24488a9cefcf16b8aae8b7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:37.405828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758120e-f055-4692-9b0b-9d340cee7e5f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:37.406207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758120e-f055-4692-9b0b-9d340cee7e5f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:37.406496 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758120e-f055-4692-9b0b-9d340cee7e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:37.425653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b758120e-f055-4692-9b0b-9d340cee7e5f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:37.426907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b758120e-f055-4692-9b0b-9d340cee7e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:37.504130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1328/2668] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:37.523046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:37.525232 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:37.620852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcf3f5-8ac1-46b0-b58a-196bbfad747d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:37.624641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154e0b-b703-44ec-9e1f-c0a7236fce52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:37.625256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154e0b-b703-44ec-9e1f-c0a7236fce52 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:37.625542 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154e0b-b703-44ec-9e1f-c0a7236fce52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:37.625820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154e0b-b703-44ec-9e1f-c0a7236fce52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:37.635298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07154e0b-b703-44ec-9e1f-c0a7236fce52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:37.636739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07154e0b-b703-44ec-9e1f-c0a7236fce52 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:37.646912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1341/2669] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:37.935044 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758120e-f055-4692-9b0b-9d340cee7e5f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:37.937579 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b758120e-f055-4692-9b0b-9d340cee7e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:37.958217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758120e-f055-4692-9b0b-9d340cee7e5f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:37.963982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:37.966204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:37.966698 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:37.967003 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] PATH_INFO: `/v3/projects/70f0d1eb8f9b45a598a02c68d0e8818c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:37.968907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] RBAC: Authorizing `identity:delete_project(project_id=70f0d1eb8f9b45a598a02c68d0e8818c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:37.971308 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:38.038587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70f0d1eb8f9b45a598a02c68d0e8818c', 'request_id': 'req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.047850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70f0d1eb8f9b45a598a02c68d0e8818c', 'request_id': 'req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.061240 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70f0d1eb8f9b45a598a02c68d0e8818c', 'request_id': 'req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.082954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] The token cache is being invalidate because project 70f0d1eb8f9b45a598a02c68d0e8818c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:38.083375 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.084521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.089672 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1329/2670] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:37 2022] DELETE /identity/v3/projects/70f0d1eb8f9b45a598a02c68d0e8818c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:38.091059 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154e0b-b703-44ec-9e1f-c0a7236fce52 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:38.103616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154e0b-b703-44ec-9e1f-c0a7236fce52 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:38.139413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:38.184793 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d4a0d-8da0-4490-bbfc-37f690d67942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:38.185439 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d4a0d-8da0-4490-bbfc-37f690d67942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:38.185718 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d4a0d-8da0-4490-bbfc-37f690d67942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:38.185990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d4a0d-8da0-4490-bbfc-37f690d67942 admin admin] PATH_INFO: `/v3/users/d41d491ae366427a8d5162ab503c7d7e` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:38.187537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d4a0d-8da0-4490-bbfc-37f690d67942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41d491ae366427a8d5162ab503c7d7e)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:38.189503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d4a0d-8da0-4490-bbfc-37f690d67942 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:38.251985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-766d4a0d-8da0-4490-bbfc-37f690d67942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41d491ae366427a8d5162ab503c7d7e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.252889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-766d4a0d-8da0-4490-bbfc-37f690d67942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41d491ae366427a8d5162ab503c7d7e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.267479 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-766d4a0d-8da0-4490-bbfc-37f690d67942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41d491ae366427a8d5162ab503c7d7e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.275998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-766d4a0d-8da0-4490-bbfc-37f690d67942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41d491ae366427a8d5162ab503c7d7e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.284613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-766d4a0d-8da0-4490-bbfc-37f690d67942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41d491ae366427a8d5162ab503c7d7e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.291281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1342/2671] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:38 2022] DELETE /identity/v3/users/d41d491ae366427a8d5162ab503c7d7e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:38.569982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:38.571611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:38.598194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6ab1a-2cfa-47b0-9628-8da0ca41b89f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:38.657819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:38.658382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:38.658673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:38.658983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] PATH_INFO: `/v3/projects/c70ae97cbbcb4253b5acc62d7650f4e2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:38.660147 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c70ae97cbbcb4253b5acc62d7650f4e2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:38.662303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:38.663977 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:38.713989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c70ae97cbbcb4253b5acc62d7650f4e2', 'request_id': 'req-6862ab34-c512-4400-99e9-d2cc3d6d6d24'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.720861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c70ae97cbbcb4253b5acc62d7650f4e2', 'request_id': 'req-6862ab34-c512-4400-99e9-d2cc3d6d6d24'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.728455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c70ae97cbbcb4253b5acc62d7650f4e2', 'request_id': 'req-6862ab34-c512-4400-99e9-d2cc3d6d6d24'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.740325 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] The token cache is being invalidate because project c70ae97cbbcb4253b5acc62d7650f4e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:38.740615 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.741243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6862ab34-c512-4400-99e9-d2cc3d6d6d24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:38.742722 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1330/2672] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:38 2022] DELETE /identity/v3/projects/c70ae97cbbcb4253b5acc62d7650f4e2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:38.747651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d39a7a-719e-46b5-b14e-8c046191f229 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:38.748168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d39a7a-719e-46b5-b14e-8c046191f229 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:38.748267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d39a7a-719e-46b5-b14e-8c046191f229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:38.767385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-72d39a7a-719e-46b5-b14e-8c046191f229 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:38.768581 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-72d39a7a-719e-46b5-b14e-8c046191f229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:38.831503 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1343/2673] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:38.839634 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d698973-77bd-4dca-be0b-8d8b5da94b8b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:38.839929 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d698973-77bd-4dca-be0b-8d8b5da94b8b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:38.840206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d698973-77bd-4dca-be0b-8d8b5da94b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:38.859940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6d698973-77bd-4dca-be0b-8d8b5da94b8b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:38.861161 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6d698973-77bd-4dca-be0b-8d8b5da94b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:38.869048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1331/2674] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:38.873259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d39a7a-719e-46b5-b14e-8c046191f229 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:38.883284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d39a7a-719e-46b5-b14e-8c046191f229 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:38.887901 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeac94-9247-4624-b60a-6477de9097f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:38.888538 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeac94-9247-4624-b60a-6477de9097f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:38.888821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeac94-9247-4624-b60a-6477de9097f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:38.889134 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeac94-9247-4624-b60a-6477de9097f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:38.890954 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aeac94-9247-4624-b60a-6477de9097f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:07:38.891264 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aeac94-9247-4624-b60a-6477de9097f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:38.892856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aeac94-9247-4624-b60a-6477de9097f0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:38.904239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1344/2675] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:07:38.909118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d698973-77bd-4dca-be0b-8d8b5da94b8b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:38.918850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d698973-77bd-4dca-be0b-8d8b5da94b8b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:38.924052 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71386d-4aae-4db0-b8b1-fa9d1fa66b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:38.924659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71386d-4aae-4db0-b8b1-fa9d1fa66b8c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:38.924939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71386d-4aae-4db0-b8b1-fa9d1fa66b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:38.925212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71386d-4aae-4db0-b8b1-fa9d1fa66b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:38.925879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71386d-4aae-4db0-b8b1-fa9d1fa66b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:38.927825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71386d-4aae-4db0-b8b1-fa9d1fa66b8c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:38.956182 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1332/2676] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:38 2022] POST /identity/v3/projects => generated 409 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:38.960382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeac94-9247-4624-b60a-6477de9097f0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:38.972142 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeac94-9247-4624-b60a-6477de9097f0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:39.009583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:39.029361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d205e10-dd2d-49ec-bcf9-d2aebe080369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:39.029975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d205e10-dd2d-49ec-bcf9-d2aebe080369 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.030885 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d205e10-dd2d-49ec-bcf9-d2aebe080369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.031163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d205e10-dd2d-49ec-bcf9-d2aebe080369 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.031915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d205e10-dd2d-49ec-bcf9-d2aebe080369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:39.033831 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d205e10-dd2d-49ec-bcf9-d2aebe080369 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:39.054352 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1345/2677] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:38 2022] POST /identity/v3/users => generated 305 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:39.058627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71386d-4aae-4db0-b8b1-fa9d1fa66b8c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:39.069089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71386d-4aae-4db0-b8b1-fa9d1fa66b8c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:39.073688 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d886a0-1139-449e-9643-293d19a762cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:39.074483 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d886a0-1139-449e-9643-293d19a762cd admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.074768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d886a0-1139-449e-9643-293d19a762cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.075078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d886a0-1139-449e-9643-293d19a762cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.075771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d886a0-1139-449e-9643-293d19a762cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:39.077894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d886a0-1139-449e-9643-293d19a762cd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:39.097044 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1333/2678] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:39.100890 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d205e10-dd2d-49ec-bcf9-d2aebe080369 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:39.112820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d205e10-dd2d-49ec-bcf9-d2aebe080369 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:39.117564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f16de-db64-4103-ac39-7f5071c8a611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:39.118178 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f16de-db64-4103-ac39-7f5071c8a611 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.118465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f16de-db64-4103-ac39-7f5071c8a611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.118763 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f16de-db64-4103-ac39-7f5071c8a611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.119488 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f16de-db64-4103-ac39-7f5071c8a611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:39.121086 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f16de-db64-4103-ac39-7f5071c8a611 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:39.131864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1346/2679] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:07:39.136195 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d886a0-1139-449e-9643-293d19a762cd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:39.147209 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d886a0-1139-449e-9643-293d19a762cd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:39.152031 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59edd3-b87e-4e46-8341-f445d8eebc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:39.152721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59edd3-b87e-4e46-8341-f445d8eebc91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.153005 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59edd3-b87e-4e46-8341-f445d8eebc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.153283 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59edd3-b87e-4e46-8341-f445d8eebc91 admin admin] PATH_INFO: `/v3/projects/ccf51b7682174496b7302ec9476f054c/users/72b7116650ce4b5193d9863a26a545b0/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.171243 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59edd3-b87e-4e46-8341-f445d8eebc91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf51b7682174496b7302ec9476f054c, user_id=72b7116650ce4b5193d9863a26a545b0, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:39.174746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59edd3-b87e-4e46-8341-f445d8eebc91 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:39.188089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1334/2680] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:39 2022] PUT /identity/v3/projects/ccf51b7682174496b7302ec9476f054c/users/72b7116650ce4b5193d9863a26a545b0/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:07:39.199662 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c095e-08a1-4a44-98c2-3d229c4b6c47 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.199958 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c095e-08a1-4a44-98c2-3d229c4b6c47 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.200237 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c095e-08a1-4a44-98c2-3d229c4b6c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.230349 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-561c095e-08a1-4a44-98c2-3d229c4b6c47 None None] MFA Rules not processed for user `72b7116650ce4b5193d9863a26a545b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:39.231617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-561c095e-08a1-4a44-98c2-3d229c4b6c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:39.303827 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1347/2681] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:39 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:39.308341 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59edd3-b87e-4e46-8341-f445d8eebc91 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:39.397649 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59edd3-b87e-4e46-8341-f445d8eebc91 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:39.401976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbe12b-1a90-461e-9319-cc737e3ab7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:39.402618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbe12b-1a90-461e-9319-cc737e3ab7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.402940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbe12b-1a90-461e-9319-cc737e3ab7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.403528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbe12b-1a90-461e-9319-cc737e3ab7f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.404488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbe12b-1a90-461e-9319-cc737e3ab7f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:39.406940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbe12b-1a90-461e-9319-cc737e3ab7f8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:39.445285 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1335/2682] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:39 2022] POST /identity/v3/projects => generated 409 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:39.448046 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c095e-08a1-4a44-98c2-3d229c4b6c47 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:39.461323 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c095e-08a1-4a44-98c2-3d229c4b6c47 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:39.531288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd4959-18be-4560-a345-284261ccd882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:39.531919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4959-18be-4560-a345-284261ccd882 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.532267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4959-18be-4560-a345-284261ccd882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.533054 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4959-18be-4560-a345-284261ccd882 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.533732 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd4959-18be-4560-a345-284261ccd882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:39.536421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd4959-18be-4560-a345-284261ccd882 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:39.562968 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1348/2683] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:39 2022] POST /identity/v3/users => generated 311 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:39.564237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbe12b-1a90-461e-9319-cc737e3ab7f8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:39.576844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbe12b-1a90-461e-9319-cc737e3ab7f8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:39.597942 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b48e89-580f-42f3-97d5-3f3833a1d5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:39.598468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b48e89-580f-42f3-97d5-3f3833a1d5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.598842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b48e89-580f-42f3-97d5-3f3833a1d5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.599352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b48e89-580f-42f3-97d5-3f3833a1d5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.600155 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b48e89-580f-42f3-97d5-3f3833a1d5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:39.601979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b48e89-580f-42f3-97d5-3f3833a1d5d0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:39.607709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:39.617340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1336/2684] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:39.618563 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd4959-18be-4560-a345-284261ccd882 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:39.630675 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd4959-18be-4560-a345-284261ccd882 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:39.640168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f24bb-1e86-449b-a5ea-13695522059d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:39.640969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f24bb-1e86-449b-a5ea-13695522059d admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.641422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f24bb-1e86-449b-a5ea-13695522059d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.641957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f24bb-1e86-449b-a5ea-13695522059d admin admin] PATH_INFO: `/v3/projects/286918fcd2784aca9972edfacb06555d/users/e6189565704e453788d1dd131a09c339/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.659830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f24bb-1e86-449b-a5ea-13695522059d admin admin] RBAC: Authorizing `identity:create_grant(project_id=286918fcd2784aca9972edfacb06555d, user_id=e6189565704e453788d1dd131a09c339, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:39.665522 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f24bb-1e86-449b-a5ea-13695522059d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:39.695637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1349/2685] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:39 2022] PUT /identity/v3/projects/286918fcd2784aca9972edfacb06555d/users/e6189565704e453788d1dd131a09c339/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:39.700022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b48e89-580f-42f3-97d5-3f3833a1d5d0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:39.711938 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b48e89-580f-42f3-97d5-3f3833a1d5d0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:39.783634 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d4ed9-94de-4fd3-8dd3-ea8976a03995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:39.784421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d4ed9-94de-4fd3-8dd3-ea8976a03995 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.784845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d4ed9-94de-4fd3-8dd3-ea8976a03995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.785126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d4ed9-94de-4fd3-8dd3-ea8976a03995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.786440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d4ed9-94de-4fd3-8dd3-ea8976a03995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:39.788719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d4ed9-94de-4fd3-8dd3-ea8976a03995 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:39.815207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1337/2686] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:39.816523 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f24bb-1e86-449b-a5ea-13695522059d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:39.828372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f24bb-1e86-449b-a5ea-13695522059d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:39.834417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40341396-846f-41fc-a7ab-236e26336442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:39.835143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40341396-846f-41fc-a7ab-236e26336442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.835430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40341396-846f-41fc-a7ab-236e26336442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.835830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40341396-846f-41fc-a7ab-236e26336442 admin admin] PATH_INFO: `/v3/projects/286918fcd2784aca9972edfacb06555d/users/e6189565704e453788d1dd131a09c339/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.837936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40341396-846f-41fc-a7ab-236e26336442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=286918fcd2784aca9972edfacb06555d, user_id=e6189565704e453788d1dd131a09c339, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:39.840792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40341396-846f-41fc-a7ab-236e26336442 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:39.848931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:39.854489 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1350/2687] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:39 2022] PUT /identity/v3/projects/286918fcd2784aca9972edfacb06555d/users/e6189565704e453788d1dd131a09c339/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:39.865978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc71ffe-73e7-4468-acaf-fb28a04d31be None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:39.866275 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc71ffe-73e7-4468-acaf-fb28a04d31be None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:39.866742 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc71ffe-73e7-4468-acaf-fb28a04d31be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:39.895993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fdc71ffe-73e7-4468-acaf-fb28a04d31be None None] MFA Rules not processed for user `e6189565704e453788d1dd131a09c339`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:39.897267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fdc71ffe-73e7-4468-acaf-fb28a04d31be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:39.969520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1338/2688] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:39 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:39.982821 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40341396-846f-41fc-a7ab-236e26336442 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:39.985583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-40341396-846f-41fc-a7ab-236e26336442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:40.044152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40341396-846f-41fc-a7ab-236e26336442 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:40.048550 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f87c03-7351-4caa-bd86-a2665d78e5f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:40.049177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f87c03-7351-4caa-bd86-a2665d78e5f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:40.049465 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f87c03-7351-4caa-bd86-a2665d78e5f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:40.049741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f87c03-7351-4caa-bd86-a2665d78e5f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:40.059366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f87c03-7351-4caa-bd86-a2665d78e5f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:40.060810 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f87c03-7351-4caa-bd86-a2665d78e5f2 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:40.071187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1351/2689] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:39 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:40.234488 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc71ffe-73e7-4468-acaf-fb28a04d31be None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:40.244133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc71ffe-73e7-4468-acaf-fb28a04d31be None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:40.247822 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd125f-74ee-480a-9fb1-84f050b5ffb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:40.248442 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd125f-74ee-480a-9fb1-84f050b5ffb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:40.248727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd125f-74ee-480a-9fb1-84f050b5ffb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:40.249724 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd125f-74ee-480a-9fb1-84f050b5ffb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:40.308422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd125f-74ee-480a-9fb1-84f050b5ffb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:40.309850 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd125f-74ee-480a-9fb1-84f050b5ffb5 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:40.319614 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1339/2690] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:40 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:42.035203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f87c03-7351-4caa-bd86-a2665d78e5f2 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:42.036870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-02f87c03-7351-4caa-bd86-a2665d78e5f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:42.126988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f87c03-7351-4caa-bd86-a2665d78e5f2 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:42.132100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91a769-2931-433d-87a6-e69ccf5de0fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:42.132733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91a769-2931-433d-87a6-e69ccf5de0fa service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:42.133012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91a769-2931-433d-87a6-e69ccf5de0fa service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:42.133284 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91a769-2931-433d-87a6-e69ccf5de0fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:42.135130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ea91a769-2931-433d-87a6-e69ccf5de0fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:42.148056 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91a769-2931-433d-87a6-e69ccf5de0fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:42.149663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91a769-2931-433d-87a6-e69ccf5de0fa service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:42.161040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1352/2691] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:47.732993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd125f-74ee-480a-9fb1-84f050b5ffb5 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:47.755454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:47.800395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd125f-74ee-480a-9fb1-84f050b5ffb5 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:47.805029 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:47.805775 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:47.806131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:47.806478 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] PATH_INFO: `/v3/users/72b7116650ce4b5193d9863a26a545b0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:47.807764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b7116650ce4b5193d9863a26a545b0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:47.809794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:47.858475 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b7116650ce4b5193d9863a26a545b0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:47.859165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b7116650ce4b5193d9863a26a545b0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:47.866918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b7116650ce4b5193d9863a26a545b0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:47.880543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b7116650ce4b5193d9863a26a545b0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:47.887617 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b7116650ce4b5193d9863a26a545b0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:47.894101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1340/2692] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:47 2022] DELETE /identity/v3/users/72b7116650ce4b5193d9863a26a545b0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:47.898706 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65753825-f1d6-4a7a-b61f-d31335a4f54e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:47.899231 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65753825-f1d6-4a7a-b61f-d31335a4f54e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:47.899374 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65753825-f1d6-4a7a-b61f-d31335a4f54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:47.919048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-65753825-f1d6-4a7a-b61f-d31335a4f54e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:47.920225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-65753825-f1d6-4a7a-b61f-d31335a4f54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:47.978139 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1353/2693] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:47.988338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:47.989748 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:48.067454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfafe8-85e7-4eb3-a5ba-7073f31ff28c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:48.071028 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74885f-9a51-47f9-be66-f3f5426062ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:48.071639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74885f-9a51-47f9-be66-f3f5426062ba service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:48.071924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74885f-9a51-47f9-be66-f3f5426062ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:48.072198 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74885f-9a51-47f9-be66-f3f5426062ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:48.081556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74885f-9a51-47f9-be66-f3f5426062ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:48.083002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74885f-9a51-47f9-be66-f3f5426062ba service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:48.092220 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1341/2694] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:48.369000 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65753825-f1d6-4a7a-b61f-d31335a4f54e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:48.370602 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-65753825-f1d6-4a7a-b61f-d31335a4f54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:48.384258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65753825-f1d6-4a7a-b61f-d31335a4f54e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:48.388888 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:48.389696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:48.390162 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:48.390453 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] PATH_INFO: `/v3/projects/ccf51b7682174496b7302ec9476f054c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:48.391676 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf51b7682174496b7302ec9476f054c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:48.393625 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:48.401998 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:48.441973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf51b7682174496b7302ec9476f054c', 'request_id': 'req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:48.447244 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf51b7682174496b7302ec9476f054c', 'request_id': 'req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:48.454030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf51b7682174496b7302ec9476f054c', 'request_id': 'req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:48.467860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] The token cache is being invalidate because project ccf51b7682174496b7302ec9476f054c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:48.468148 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:48.468803 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4e0c0e7f-8ab4-4c4b-964e-4d0759d30a21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:48.470741 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1354/2695] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:48 2022] DELETE /identity/v3/projects/ccf51b7682174496b7302ec9476f054c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:48.474770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74885f-9a51-47f9-be66-f3f5426062ba service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:48.484882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74885f-9a51-47f9-be66-f3f5426062ba service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:48.504076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:48.535562 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce2b49-2907-45c4-9b76-06d6d2a2912b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:48.536113 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce2b49-2907-45c4-9b76-06d6d2a2912b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:48.536393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce2b49-2907-45c4-9b76-06d6d2a2912b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:48.550443 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:48.551216 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:48.551750 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:48.552202 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] PATH_INFO: `/v3/users/e6189565704e453788d1dd131a09c339` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:48.553586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6189565704e453788d1dd131a09c339)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:48.555618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:48.556332 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9cce2b49-2907-45c4-9b76-06d6d2a2912b None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:48.557952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9cce2b49-2907-45c4-9b76-06d6d2a2912b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:48.602397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6189565704e453788d1dd131a09c339'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:48.603082 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6189565704e453788d1dd131a09c339'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:48.612372 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6189565704e453788d1dd131a09c339'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:48.616260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1355/2696] 10.222.0.46 () {58 vars in 967 bytes} [Thu Jul 28 04:07:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:07:48.625386 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6189565704e453788d1dd131a09c339'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:48.629040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce2b49-2907-45c4-9b76-06d6d2a2912b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:48.630611 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9cce2b49-2907-45c4-9b76-06d6d2a2912b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:48.632464 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6189565704e453788d1dd131a09c339'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:48.638333 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1342/2697] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:48 2022] DELETE /identity/v3/users/e6189565704e453788d1dd131a09c339 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:48.711425 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce2b49-2907-45c4-9b76-06d6d2a2912b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:48.761936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:48.788699 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe1da3-2a14-497d-8829-a563d8e70344 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:48.790024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe1da3-2a14-497d-8829-a563d8e70344 service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:48.790024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe1da3-2a14-497d-8829-a563d8e70344 service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:48.790024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe1da3-2a14-497d-8829-a563d8e70344 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:48.791763 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1bfe1da3-2a14-497d-8829-a563d8e70344 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:48.887246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe1da3-2a14-497d-8829-a563d8e70344 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:48.889571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe1da3-2a14-497d-8829-a563d8e70344 service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:48.907075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1356/2698] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:07:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 04:07:48.972218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:48.974273 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:49.082611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fa377-55a1-4991-a27a-0a7a84f69e84 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:49.088701 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:49.089448 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:49.089749 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:49.089885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] PATH_INFO: `/v3/projects/286918fcd2784aca9972edfacb06555d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:49.091166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=286918fcd2784aca9972edfacb06555d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:49.093082 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:49.159264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:49.165357 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '286918fcd2784aca9972edfacb06555d', 'request_id': 'req-3529ae47-431a-49e3-8517-c70e2bf2cba2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:49.175711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '286918fcd2784aca9972edfacb06555d', 'request_id': 'req-3529ae47-431a-49e3-8517-c70e2bf2cba2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:49.183939 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '286918fcd2784aca9972edfacb06555d', 'request_id': 'req-3529ae47-431a-49e3-8517-c70e2bf2cba2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:49.197683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] The token cache is being invalidate because project 286918fcd2784aca9972edfacb06555d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:49.197888 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:49.198575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3529ae47-431a-49e3-8517-c70e2bf2cba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:49.203068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1343/2699] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:48 2022] DELETE /identity/v3/projects/286918fcd2784aca9972edfacb06555d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:49.205859 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f836-c856-41ac-b50e-428dee93e8e7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:49.206151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f836-c856-41ac-b50e-428dee93e8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:49.206620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f836-c856-41ac-b50e-428dee93e8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:49.226312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f9a0f836-c856-41ac-b50e-428dee93e8e7 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:49.227577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f9a0f836-c856-41ac-b50e-428dee93e8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:49.302898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1357/2700] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:07:49.311435 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e7b7f-e8c7-4a0c-b330-64e42ba35bf5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:49.311928 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e7b7f-e8c7-4a0c-b330-64e42ba35bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:49.312279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e7b7f-e8c7-4a0c-b330-64e42ba35bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:49.333789 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-236e7b7f-e8c7-4a0c-b330-64e42ba35bf5 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:49.335030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-236e7b7f-e8c7-4a0c-b330-64e42ba35bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:49.343097 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1344/2701] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:49.347433 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0f836-c856-41ac-b50e-428dee93e8e7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:49.357678 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0f836-c856-41ac-b50e-428dee93e8e7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:49.363025 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c451957-0edc-4da6-9ba1-83d67c250eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:49.363643 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451957-0edc-4da6-9ba1-83d67c250eca admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:49.363957 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451957-0edc-4da6-9ba1-83d67c250eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:49.364238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451957-0edc-4da6-9ba1-83d67c250eca admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:49.364942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c451957-0edc-4da6-9ba1-83d67c250eca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:07:49.365258 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c451957-0edc-4da6-9ba1-83d67c250eca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:49.366912 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c451957-0edc-4da6-9ba1-83d67c250eca admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:49.380991 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1358/2702] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:07:49.385329 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e7b7f-e8c7-4a0c-b330-64e42ba35bf5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:49.396031 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e7b7f-e8c7-4a0c-b330-64e42ba35bf5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:49.400696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7be4d3-e05a-42e3-9211-e3d3fda63e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:49.401303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7be4d3-e05a-42e3-9211-e3d3fda63e24 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:49.401584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7be4d3-e05a-42e3-9211-e3d3fda63e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:49.401857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7be4d3-e05a-42e3-9211-e3d3fda63e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:49.402527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7be4d3-e05a-42e3-9211-e3d3fda63e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:49.404519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7be4d3-e05a-42e3-9211-e3d3fda63e24 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:49.434541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1345/2703] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:07:49 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:07:49.439291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c451957-0edc-4da6-9ba1-83d67c250eca admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:49.449209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c451957-0edc-4da6-9ba1-83d67c250eca admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:49.508478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7ccaf-62fc-4b55-a9e2-4fc534559e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:49.509095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ccaf-62fc-4b55-a9e2-4fc534559e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:49.509515 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ccaf-62fc-4b55-a9e2-4fc534559e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:49.509740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ccaf-62fc-4b55-a9e2-4fc534559e41 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:49.510310 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7ccaf-62fc-4b55-a9e2-4fc534559e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:49.512314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7ccaf-62fc-4b55-a9e2-4fc534559e41 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:49.534412 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1359/2704] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:07:49 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:07:49.539503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7be4d3-e05a-42e3-9211-e3d3fda63e24 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:49.552410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7be4d3-e05a-42e3-9211-e3d3fda63e24 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:49.557170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14687bb-2d7a-44bd-b994-6bc514b55ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:49.557899 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14687bb-2d7a-44bd-b994-6bc514b55ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:49.558250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14687bb-2d7a-44bd-b994-6bc514b55ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:49.558588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14687bb-2d7a-44bd-b994-6bc514b55ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:49.559393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14687bb-2d7a-44bd-b994-6bc514b55ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:49.561657 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14687bb-2d7a-44bd-b994-6bc514b55ae6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:49.572489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1346/2705] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:49.576334 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7ccaf-62fc-4b55-a9e2-4fc534559e41 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:49.581882 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:49.588696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7ccaf-62fc-4b55-a9e2-4fc534559e41 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:49.594570 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a69c9-aba6-48c6-8e2a-68c75e30f712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:49.595251 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a69c9-aba6-48c6-8e2a-68c75e30f712 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:49.595531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a69c9-aba6-48c6-8e2a-68c75e30f712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:49.595803 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a69c9-aba6-48c6-8e2a-68c75e30f712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:49.596557 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a69c9-aba6-48c6-8e2a-68c75e30f712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:49.598158 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a69c9-aba6-48c6-8e2a-68c75e30f712 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:49.610580 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1360/2706] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:07:49.614825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14687bb-2d7a-44bd-b994-6bc514b55ae6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:49.625279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14687bb-2d7a-44bd-b994-6bc514b55ae6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:49.629960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9be845-1ec9-4f82-95e3-d1cbecd9b22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:49.630719 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9be845-1ec9-4f82-95e3-d1cbecd9b22b admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:49.631091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9be845-1ec9-4f82-95e3-d1cbecd9b22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:49.631236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9be845-1ec9-4f82-95e3-d1cbecd9b22b admin admin] PATH_INFO: `/v3/projects/96a95920253749599d292eac5cdcdd59/users/2c058b7ed8204341822066ebb96d2379/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:49.648081 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9be845-1ec9-4f82-95e3-d1cbecd9b22b admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a95920253749599d292eac5cdcdd59, user_id=2c058b7ed8204341822066ebb96d2379, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:49.650685 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9be845-1ec9-4f82-95e3-d1cbecd9b22b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:49.663921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1347/2707] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:07:49 2022] PUT /identity/v3/projects/96a95920253749599d292eac5cdcdd59/users/2c058b7ed8204341822066ebb96d2379/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:07:49.676843 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b977e-52cd-4176-8bbd-a508d64394f5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:49.677192 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b977e-52cd-4176-8bbd-a508d64394f5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:49.677493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b977e-52cd-4176-8bbd-a508d64394f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:49.705952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-108b977e-52cd-4176-8bbd-a508d64394f5 None None] MFA Rules not processed for user `2c058b7ed8204341822066ebb96d2379`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:49.707224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-108b977e-52cd-4176-8bbd-a508d64394f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:49.771505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1361/2708] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:07:49.783796 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9be845-1ec9-4f82-95e3-d1cbecd9b22b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:49.785501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4a9be845-1ec9-4f82-95e3-d1cbecd9b22b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:49.842647 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9be845-1ec9-4f82-95e3-d1cbecd9b22b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:49.846349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e52564-6708-4a07-94c7-8340a8551847 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:49.846995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e52564-6708-4a07-94c7-8340a8551847 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:49.847279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e52564-6708-4a07-94c7-8340a8551847 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:49.847547 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e52564-6708-4a07-94c7-8340a8551847 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:49.855737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e52564-6708-4a07-94c7-8340a8551847 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:49.857140 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e52564-6708-4a07-94c7-8340a8551847 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:49.866412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1348/2709] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:56.927049 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b977e-52cd-4176-8bbd-a508d64394f5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:56.990123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b977e-52cd-4176-8bbd-a508d64394f5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:56.995154 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:56.995789 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:56.996074 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:56.996352 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] PATH_INFO: `/v3/users/2c058b7ed8204341822066ebb96d2379` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:56.997516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c058b7ed8204341822066ebb96d2379)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:57.000187 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:57.050960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c058b7ed8204341822066ebb96d2379'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:57.051618 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c058b7ed8204341822066ebb96d2379'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:57.063423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c058b7ed8204341822066ebb96d2379'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:57.070168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c058b7ed8204341822066ebb96d2379'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:57.077403 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c058b7ed8204341822066ebb96d2379'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:57.079568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:57.084534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1362/2710] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:07:56 2022] DELETE /identity/v3/users/2c058b7ed8204341822066ebb96d2379 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:07:57.089254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071ee83-46c3-4463-afa8-7219de457f2c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:57.089710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071ee83-46c3-4463-afa8-7219de457f2c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:57.089808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071ee83-46c3-4463-afa8-7219de457f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:57.110256 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6071ee83-46c3-4463-afa8-7219de457f2c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:07:57.111501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6071ee83-46c3-4463-afa8-7219de457f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:57.171289 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1349/2711] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:07:57.180818 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:57.182220 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:57.239747 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28b1e8-4ef9-4c1e-94cc-6e3ed1f1c6da admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:57.243382 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b6d34-6447-436b-aad0-19105fa5b83c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:57.243997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b6d34-6447-436b-aad0-19105fa5b83c service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:57.244280 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b6d34-6447-436b-aad0-19105fa5b83c service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:57.244579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b6d34-6447-436b-aad0-19105fa5b83c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:57.253715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b6d34-6447-436b-aad0-19105fa5b83c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:57.255190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b6d34-6447-436b-aad0-19105fa5b83c service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:57.265121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1363/2712] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:07:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:07:57.527339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071ee83-46c3-4463-afa8-7219de457f2c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:07:57.528915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6071ee83-46c3-4463-afa8-7219de457f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:07:57.542224 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071ee83-46c3-4463-afa8-7219de457f2c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:07:57.547580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:07:57.548223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:07:57.548507 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:07:57.548778 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] PATH_INFO: `/v3/projects/96a95920253749599d292eac5cdcdd59` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:07:57.550113 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a95920253749599d292eac5cdcdd59)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:07:57.551955 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:07:57.559278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:07:57.601298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a95920253749599d292eac5cdcdd59', 'request_id': 'req-f1fdd93b-af0e-497f-aea5-22310c35e8c6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:57.606663 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a95920253749599d292eac5cdcdd59', 'request_id': 'req-f1fdd93b-af0e-497f-aea5-22310c35e8c6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:57.615408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a95920253749599d292eac5cdcdd59', 'request_id': 'req-f1fdd93b-af0e-497f-aea5-22310c35e8c6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:57.626291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] The token cache is being invalidate because project 96a95920253749599d292eac5cdcdd59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:07:57.626577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:57.627228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:07:57.628721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1350/2713] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:07:57 2022] DELETE /identity/v3/projects/96a95920253749599d292eac5cdcdd59 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:08:02.022889 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b6d34-6447-436b-aad0-19105fa5b83c service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:08:02.024346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1a5b6d34-6447-436b-aad0-19105fa5b83c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:08:02.036008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b6d34-6447-436b-aad0-19105fa5b83c service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:08:02.085330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516094d9-4205-4a0f-8f0c-68870ac18cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:08:02.085933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516094d9-4205-4a0f-8f0c-68870ac18cce service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:08:02.086217 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516094d9-4205-4a0f-8f0c-68870ac18cce service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:08:02.086490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516094d9-4205-4a0f-8f0c-68870ac18cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:08:02.088247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-516094d9-4205-4a0f-8f0c-68870ac18cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:08:02.149739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516094d9-4205-4a0f-8f0c-68870ac18cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:08:02.151200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516094d9-4205-4a0f-8f0c-68870ac18cce service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:08:02.160589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1364/2714] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:08:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:08:07.623584 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:08:08.885754 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:08:08.887285 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:08:08.964608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdd93b-af0e-497f-aea5-22310c35e8c6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:08:08.969076 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e8326-def9-4738-931e-f1fc9735951e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:08:08.969681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e8326-def9-4738-931e-f1fc9735951e service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:08:08.969963 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e8326-def9-4738-931e-f1fc9735951e service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:08:08.970237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e8326-def9-4738-931e-f1fc9735951e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:08:08.971977 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a85e8326-def9-4738-931e-f1fc9735951e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:08:09.028844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e8326-def9-4738-931e-f1fc9735951e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:08:09.031294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e8326-def9-4738-931e-f1fc9735951e service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:08:09.040447 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1351/2715] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:08:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:08:12.156380 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=76 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:08:16.945482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db935ab3-2682-4272-bd44-b5754c317a2c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:08:16.946084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db935ab3-2682-4272-bd44-b5754c317a2c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:08:16.946084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db935ab3-2682-4272-bd44-b5754c317a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:08:16.965152 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-db935ab3-2682-4272-bd44-b5754c317a2c None None] MFA Rules not processed for user `352d60372ba249e38ea4aa2d62723f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:08:16.967044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-db935ab3-2682-4272-bd44-b5754c317a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:08:16.973955 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1365/2716] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:08:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:08:19.036383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=57 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:08:26.959855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:08:37.612885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e8326-def9-4738-931e-f1fc9735951e service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:08:37.615145 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a85e8326-def9-4738-931e-f1fc9735951e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:08:37.735114 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e8326-def9-4738-931e-f1fc9735951e service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:08:37.742023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170d486-51f6-467a-8205-d92e43b492ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:08:37.742824 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170d486-51f6-467a-8205-d92e43b492ce service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:08:37.743212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170d486-51f6-467a-8205-d92e43b492ce service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:08:37.743684 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170d486-51f6-467a-8205-d92e43b492ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:08:37.745745 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9170d486-51f6-467a-8205-d92e43b492ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:08:37.763830 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170d486-51f6-467a-8205-d92e43b492ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:08:37.765708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170d486-51f6-467a-8205-d92e43b492ce service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:08:37.781788 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1352/2717] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:08:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:08:38.634190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de88e1-2025-4e35-afc7-2488b2df6b2a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:08:38.635129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de88e1-2025-4e35-afc7-2488b2df6b2a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:08:38.635572 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de88e1-2025-4e35-afc7-2488b2df6b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:08:38.658137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c0de88e1-2025-4e35-afc7-2488b2df6b2a None None] MFA Rules not processed for user `b7e6ceb0f76d41ce962efd065ffa3d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:08:38.660333 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c0de88e1-2025-4e35-afc7-2488b2df6b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:08:38.729714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1366/2718] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:08:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:08:47.774572 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=43 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:08:48.726740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=25 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:09:30.482565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170d486-51f6-467a-8205-d92e43b492ce service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:09:30.484523 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9170d486-51f6-467a-8205-d92e43b492ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:09:30.495795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170d486-51f6-467a-8205-d92e43b492ce service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:09:30.499453 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78da6e-f758-4133-b92a-7247dd0199d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:09:30.500668 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78da6e-f758-4133-b92a-7247dd0199d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:09:30.500951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78da6e-f758-4133-b92a-7247dd0199d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:09:30.501225 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78da6e-f758-4133-b92a-7247dd0199d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:09:30.502997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ab78da6e-f758-4133-b92a-7247dd0199d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:09:30.516129 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78da6e-f758-4133-b92a-7247dd0199d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:09:30.517725 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78da6e-f758-4133-b92a-7247dd0199d2 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:09:30.529254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1353/2719] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:09:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:09:32.862176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c887df6-ba68-4cee-8924-2a9358c0d35a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:09:32.862635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c887df6-ba68-4cee-8924-2a9358c0d35a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:09:32.862743 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c887df6-ba68-4cee-8924-2a9358c0d35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:09:32.882559 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1c887df6-ba68-4cee-8924-2a9358c0d35a None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:09:32.883788 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1c887df6-ba68-4cee-8924-2a9358c0d35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:09:32.940292 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1367/2720] 10.222.0.46 () {58 vars in 967 bytes} [Thu Jul 28 04:09:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:09:32.986552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78da6e-f758-4133-b92a-7247dd0199d2 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:09:32.987979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ab78da6e-f758-4133-b92a-7247dd0199d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:09:33.057126 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78da6e-f758-4133-b92a-7247dd0199d2 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:09:33.062011 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69728507-e291-4a57-a72f-6ca7c4fd1b15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:09:33.063610 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69728507-e291-4a57-a72f-6ca7c4fd1b15 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:09:33.063897 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69728507-e291-4a57-a72f-6ca7c4fd1b15 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:09:33.064173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69728507-e291-4a57-a72f-6ca7c4fd1b15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:09:33.072091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69728507-e291-4a57-a72f-6ca7c4fd1b15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:09:33.073666 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69728507-e291-4a57-a72f-6ca7c4fd1b15 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:09:33.082633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1354/2721] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:09:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:09:39.486959 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c887df6-ba68-4cee-8924-2a9358c0d35a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:09:39.488929 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1c887df6-ba68-4cee-8924-2a9358c0d35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:09:39.562820 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c887df6-ba68-4cee-8924-2a9358c0d35a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:09:39.567739 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:09:39.568381 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:09:39.568661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:09:39.568932 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] PATH_INFO: `/v3/users/9d30603fefe44a0c9dbed8c170071b09` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:09:39.570134 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d30603fefe44a0c9dbed8c170071b09)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:09:39.572064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:09:39.621922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d30603fefe44a0c9dbed8c170071b09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:39.623291 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d30603fefe44a0c9dbed8c170071b09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:39.637161 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d30603fefe44a0c9dbed8c170071b09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:39.647342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d30603fefe44a0c9dbed8c170071b09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:39.654816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d30603fefe44a0c9dbed8c170071b09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:39.662493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1368/2722] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:09:39 2022] DELETE /identity/v3/users/9d30603fefe44a0c9dbed8c170071b09 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:09:39.838439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69728507-e291-4a57-a72f-6ca7c4fd1b15 service swift] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:09:39.850512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69728507-e291-4a57-a72f-6ca7c4fd1b15 service swift] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:09:39.908624 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:09:39.909194 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:09:39.909480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:09:39.909778 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] PATH_INFO: `/v3/projects/346eb7e46ec54cf087d8b39379cf5129` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:09:39.910989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=346eb7e46ec54cf087d8b39379cf5129)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:09:39.913201 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:09:39.960466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '346eb7e46ec54cf087d8b39379cf5129', 'request_id': 'req-c66b318d-1864-43ac-80e3-a1b65e85d273'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:39.965908 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '346eb7e46ec54cf087d8b39379cf5129', 'request_id': 'req-c66b318d-1864-43ac-80e3-a1b65e85d273'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:39.973670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '346eb7e46ec54cf087d8b39379cf5129', 'request_id': 'req-c66b318d-1864-43ac-80e3-a1b65e85d273'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:39.984556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] The token cache is being invalidate because project 346eb7e46ec54cf087d8b39379cf5129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:09:39.984840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:39.985468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:39.986997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1355/2723] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:09:39 2022] DELETE /identity/v3/projects/346eb7e46ec54cf087d8b39379cf5129 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:09:39.991072 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:09:39.992589 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:09:40.005750 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9581fe-64cc-4091-a90b-d84b9512f129 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:09:40.025803 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:09:40.063641 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:09:40.064281 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:09:40.064564 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:09:40.064841 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] PATH_INFO: `/v3/users/c992bd261bdb418cb1a4b604a8e1bd8d` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:09:40.066007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c992bd261bdb418cb1a4b604a8e1bd8d)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:09:40.067951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:09:40.113063 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c992bd261bdb418cb1a4b604a8e1bd8d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:40.113708 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c992bd261bdb418cb1a4b604a8e1bd8d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:40.125771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c992bd261bdb418cb1a4b604a8e1bd8d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:40.132694 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c992bd261bdb418cb1a4b604a8e1bd8d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:40.141013 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c992bd261bdb418cb1a4b604a8e1bd8d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:40.146859 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1369/2724] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:09:39 2022] DELETE /identity/v3/users/c992bd261bdb418cb1a4b604a8e1bd8d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:09:40.319972 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:09:40.321460 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:09:40.339835 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b318d-1864-43ac-80e3-a1b65e85d273 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:09:40.344230 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:09:40.400486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:09:40.401135 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:09:40.401421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:09:40.401708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] PATH_INFO: `/v3/projects/cdf6f41af5c34ec9b449157e36c4b372` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:09:40.403095 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf6f41af5c34ec9b449157e36c4b372)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:09:40.405236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:09:40.451383 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf6f41af5c34ec9b449157e36c4b372', 'request_id': 'req-401bc126-035d-4e2f-89fc-b879955dd093'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:40.457171 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf6f41af5c34ec9b449157e36c4b372', 'request_id': 'req-401bc126-035d-4e2f-89fc-b879955dd093'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:40.464290 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf6f41af5c34ec9b449157e36c4b372', 'request_id': 'req-401bc126-035d-4e2f-89fc-b879955dd093'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:40.475900 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] The token cache is being invalidate because project cdf6f41af5c34ec9b449157e36c4b372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:09:40.476187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:40.476821 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:09:40.478354 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1356/2725] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:09:40 2022] DELETE /identity/v3/projects/cdf6f41af5c34ec9b449157e36c4b372 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:09:50.143428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:09:50.472786 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:09:56.201544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:09:56.202993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:09:56.259599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b4381-580f-49c6-b1ae-e416f55549b0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:09:56.306421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535cb82-b8eb-46f1-b91d-053b98cfeca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:09:56.307058 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535cb82-b8eb-46f1-b91d-053b98cfeca4 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:09:56.307342 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535cb82-b8eb-46f1-b91d-053b98cfeca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:09:56.307615 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535cb82-b8eb-46f1-b91d-053b98cfeca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:09:56.309346 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7535cb82-b8eb-46f1-b91d-053b98cfeca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:09:56.377845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535cb82-b8eb-46f1-b91d-053b98cfeca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:09:56.379724 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535cb82-b8eb-46f1-b91d-053b98cfeca4 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:09:56.391684 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1370/2726] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:09:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:10:06.386168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=74 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:10:06.397296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:06.398971 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:06.413054 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bc126-035d-4e2f-89fc-b879955dd093 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:06.416991 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a010a7-5520-4588-9f65-8d11191a7a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:06.417651 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a010a7-5520-4588-9f65-8d11191a7a49 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:06.418030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a010a7-5520-4588-9f65-8d11191a7a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:06.418995 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a010a7-5520-4588-9f65-8d11191a7a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:06.420911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-01a010a7-5520-4588-9f65-8d11191a7a49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:06.436510 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a010a7-5520-4588-9f65-8d11191a7a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:06.450065 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a010a7-5520-4588-9f65-8d11191a7a49 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:06.464582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1357/2727] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:10:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:10:08.904667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535cb82-b8eb-46f1-b91d-053b98cfeca4 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:08.906658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7535cb82-b8eb-46f1-b91d-053b98cfeca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:08.921285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535cb82-b8eb-46f1-b91d-053b98cfeca4 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:08.929067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067cb1bb-92c3-4f4d-aac2-aa75d18febc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:08.929697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cb1bb-92c3-4f4d-aac2-aa75d18febc9 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:08.930007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cb1bb-92c3-4f4d-aac2-aa75d18febc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:08.930285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cb1bb-92c3-4f4d-aac2-aa75d18febc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:08.932078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-067cb1bb-92c3-4f4d-aac2-aa75d18febc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:08.945308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067cb1bb-92c3-4f4d-aac2-aa75d18febc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:08.946966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067cb1bb-92c3-4f4d-aac2-aa75d18febc9 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:08.958530 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1371/2728] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:10:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:10:16.457948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:10:18.611308 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a010a7-5520-4588-9f65-8d11191a7a49 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:18.612864 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-01a010a7-5520-4588-9f65-8d11191a7a49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:18.623298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a010a7-5520-4588-9f65-8d11191a7a49 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:18.627533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e993f-7ea9-45db-988b-fc5c1ca92f9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:18.628137 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e993f-7ea9-45db-988b-fc5c1ca92f9a service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:18.628419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e993f-7ea9-45db-988b-fc5c1ca92f9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:18.628693 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e993f-7ea9-45db-988b-fc5c1ca92f9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:18.630371 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-428e993f-7ea9-45db-988b-fc5c1ca92f9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:18.692994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e993f-7ea9-45db-988b-fc5c1ca92f9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:18.694417 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e993f-7ea9-45db-988b-fc5c1ca92f9a service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:18.703923 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1358/2729] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:10:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:10:18.953180 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:10:23.659225 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed57a94-a6bc-497d-a69c-edf627294bdd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:23.659867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed57a94-a6bc-497d-a69c-edf627294bdd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:23.659867 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed57a94-a6bc-497d-a69c-edf627294bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:23.724057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fed57a94-a6bc-497d-a69c-edf627294bdd None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:10:23.725472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fed57a94-a6bc-497d-a69c-edf627294bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:23.810241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1372/2730] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 04:10:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:10:23.818689 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e993f-7ea9-45db-988b-fc5c1ca92f9a service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:23.820870 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-428e993f-7ea9-45db-988b-fc5c1ca92f9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:23.942012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e993f-7ea9-45db-988b-fc5c1ca92f9a service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:23.947968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a35016-a46f-494b-9b1e-8e2822e863ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:23.948960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a35016-a46f-494b-9b1e-8e2822e863ea service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:23.949280 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a35016-a46f-494b-9b1e-8e2822e863ea service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:23.949565 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a35016-a46f-494b-9b1e-8e2822e863ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:23.961277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a35016-a46f-494b-9b1e-8e2822e863ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:23.962984 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a35016-a46f-494b-9b1e-8e2822e863ea service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:23.979074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1359/2731] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:10:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 04:10:25.585943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed57a94-a6bc-497d-a69c-edf627294bdd None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:25.587689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fed57a94-a6bc-497d-a69c-edf627294bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:25.604996 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed57a94-a6bc-497d-a69c-edf627294bdd None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:25.610103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:25.611466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:25.611965 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:25.612653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] PATH_INFO: `/v3/users/352d60372ba249e38ea4aa2d62723f19` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:25.614497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=352d60372ba249e38ea4aa2d62723f19)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:25.618524 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:25.712215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '352d60372ba249e38ea4aa2d62723f19'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:25.713255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '352d60372ba249e38ea4aa2d62723f19'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:25.755622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '352d60372ba249e38ea4aa2d62723f19'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:25.782257 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '352d60372ba249e38ea4aa2d62723f19'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:25.802362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '352d60372ba249e38ea4aa2d62723f19'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:25.814441 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1373/2732] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:10:25 2022] DELETE /identity/v3/users/352d60372ba249e38ea4aa2d62723f19 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:10:26.022913 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a35016-a46f-494b-9b1e-8e2822e863ea service swift] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:26.052262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a35016-a46f-494b-9b1e-8e2822e863ea service swift] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:26.140056 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:26.140710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:26.140988 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:26.141262 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] PATH_INFO: `/v3/projects/338679f381ed4e9e898a59289b0f8178` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:26.142503 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=338679f381ed4e9e898a59289b0f8178)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:26.144573 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:26.157871 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:10:26.215704 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '338679f381ed4e9e898a59289b0f8178', 'request_id': 'req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.223830 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '338679f381ed4e9e898a59289b0f8178', 'request_id': 'req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.236044 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '338679f381ed4e9e898a59289b0f8178', 'request_id': 'req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.251026 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] The token cache is being invalidate because project 338679f381ed4e9e898a59289b0f8178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:10:26.251780 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.252718 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.254490 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1360/2733] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:10:26 2022] DELETE /identity/v3/projects/338679f381ed4e9e898a59289b0f8178 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:10:26.258525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:26.260003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:26.278597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f40ee-6aa0-496b-adac-dd68f26c95ce admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:26.328582 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:10:26.348639 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:26.349391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:26.349783 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:26.350129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] PATH_INFO: `/v3/users/ac14f1b5d01741468a111a80c72e2763` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:26.351407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac14f1b5d01741468a111a80c72e2763)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:26.353385 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:26.417907 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac14f1b5d01741468a111a80c72e2763'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.419330 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac14f1b5d01741468a111a80c72e2763'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.434683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac14f1b5d01741468a111a80c72e2763'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.448039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac14f1b5d01741468a111a80c72e2763'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.456919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac14f1b5d01741468a111a80c72e2763'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.464576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1374/2734] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:10:26 2022] DELETE /identity/v3/users/ac14f1b5d01741468a111a80c72e2763 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:10:26.677142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:26.678823 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:26.703500 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b89ec9-596b-41e3-bb05-acd55bf85bb8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:26.795223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:26.795979 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:26.796363 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:26.796564 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] PATH_INFO: `/v3/projects/e7aed6d258a0490a97f4ffc6de84193b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:26.798179 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7aed6d258a0490a97f4ffc6de84193b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:26.800117 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:26.887629 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7aed6d258a0490a97f4ffc6de84193b', 'request_id': 'req-df490be7-8cee-4fc5-b3c0-3667aa15c54d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.895832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7aed6d258a0490a97f4ffc6de84193b', 'request_id': 'req-df490be7-8cee-4fc5-b3c0-3667aa15c54d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.904834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7aed6d258a0490a97f4ffc6de84193b', 'request_id': 'req-df490be7-8cee-4fc5-b3c0-3667aa15c54d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.918936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] The token cache is being invalidate because project e7aed6d258a0490a97f4ffc6de84193b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:10:26.919598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.920305 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:10:26.922146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1361/2735] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:10:26 2022] DELETE /identity/v3/projects/e7aed6d258a0490a97f4ffc6de84193b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:10:36.460165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:10:36.919707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=73,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:10:37.026552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:37.028007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:37.086269 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3276f59-8d70-4d0a-91e1-0b5f2f33492f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:37.134095 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f2d08-5398-4c6c-9967-832eb2f5fc8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:37.134753 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f2d08-5398-4c6c-9967-832eb2f5fc8d service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:37.135104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f2d08-5398-4c6c-9967-832eb2f5fc8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:37.135387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f2d08-5398-4c6c-9967-832eb2f5fc8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:37.138315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-956f2d08-5398-4c6c-9967-832eb2f5fc8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:37.196975 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f2d08-5398-4c6c-9967-832eb2f5fc8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:37.198408 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f2d08-5398-4c6c-9967-832eb2f5fc8d service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:37.207627 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1375/2736] 10.222.0.46 () {60 vars in 1396 bytes} [Thu Jul 28 04:10:37 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:10:37.430212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:37.431669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:37.488618 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df490be7-8cee-4fc5-b3c0-3667aa15c54d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:37.492270 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf55e5b-9255-4373-9a3f-00e9a15d30ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:37.492882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf55e5b-9255-4373-9a3f-00e9a15d30ff service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:37.493168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf55e5b-9255-4373-9a3f-00e9a15d30ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:37.493446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf55e5b-9255-4373-9a3f-00e9a15d30ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:37.495227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bdf55e5b-9255-4373-9a3f-00e9a15d30ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:37.504673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf55e5b-9255-4373-9a3f-00e9a15d30ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:37.506089 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf55e5b-9255-4373-9a3f-00e9a15d30ff service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:37.514023 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1362/2737] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:10:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:10:37.655990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f2d08-5398-4c6c-9967-832eb2f5fc8d service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:37.658071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-956f2d08-5398-4c6c-9967-832eb2f5fc8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:37.744658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f2d08-5398-4c6c-9967-832eb2f5fc8d service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:37.751676 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a965e-bc65-432d-b114-8a1a3606dd71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:37.752398 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a965e-bc65-432d-b114-8a1a3606dd71 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:37.752779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a965e-bc65-432d-b114-8a1a3606dd71 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:37.753147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a965e-bc65-432d-b114-8a1a3606dd71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:37.755129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-404a965e-bc65-432d-b114-8a1a3606dd71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:37.770774 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a965e-bc65-432d-b114-8a1a3606dd71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:37.772691 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a965e-bc65-432d-b114-8a1a3606dd71 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:37.780145 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:10:37.796723 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1376/2738] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:10:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:10:37.874424 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf55e5b-9255-4373-9a3f-00e9a15d30ff service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:37.876583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bdf55e5b-9255-4373-9a3f-00e9a15d30ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:37.944919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf55e5b-9255-4373-9a3f-00e9a15d30ff service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:37.949815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588a859-b14d-4351-9ecd-801c382cbe43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:37.950418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588a859-b14d-4351-9ecd-801c382cbe43 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:37.950717 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588a859-b14d-4351-9ecd-801c382cbe43 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:37.950999 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588a859-b14d-4351-9ecd-801c382cbe43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:37.952714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6588a859-b14d-4351-9ecd-801c382cbe43 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:37.964013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588a859-b14d-4351-9ecd-801c382cbe43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:37.965603 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588a859-b14d-4351-9ecd-801c382cbe43 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:37.975411 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1363/2739] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:10:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:10:39.495003 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffad9f-0d0c-4251-a55f-3ddb909f1227 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:39.495605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffad9f-0d0c-4251-a55f-3ddb909f1227 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:39.495605 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffad9f-0d0c-4251-a55f-3ddb909f1227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:39.518442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b7ffad9f-0d0c-4251-a55f-3ddb909f1227 None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:10:39.519728 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b7ffad9f-0d0c-4251-a55f-3ddb909f1227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:39.598683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1377/2740] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jul 28 04:10:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:10:39.606684 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588a859-b14d-4351-9ecd-801c382cbe43 service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:39.608327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6588a859-b14d-4351-9ecd-801c382cbe43 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:39.686539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588a859-b14d-4351-9ecd-801c382cbe43 service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:39.692421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb736c-8fcd-4855-aa2a-c97051d93e3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:39.693192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb736c-8fcd-4855-aa2a-c97051d93e3a service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:39.693479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb736c-8fcd-4855-aa2a-c97051d93e3a service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:39.693759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb736c-8fcd-4855-aa2a-c97051d93e3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:39.702281 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb736c-8fcd-4855-aa2a-c97051d93e3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:39.703949 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb736c-8fcd-4855-aa2a-c97051d93e3a service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:39.712393 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1364/2741] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:10:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:10:49.585898 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=27 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:10:49.708950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:10:53.362790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5854fc1-f659-4893-8d10-e508f62a6b84 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:53.363289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5854fc1-f659-4893-8d10-e508f62a6b84 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:53.363399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5854fc1-f659-4893-8d10-e508f62a6b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:53.383307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d5854fc1-f659-4893-8d10-e508f62a6b84 None None] MFA Rules not processed for user `3b23317779d94f99bf804a172bb9c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:10:53.385039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d5854fc1-f659-4893-8d10-e508f62a6b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:10:53.393050 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1378/2742] 10.222.0.46 () {58 vars in 967 bytes} [Thu Jul 28 04:10:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:10:53.405173 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb736c-8fcd-4855-aa2a-c97051d93e3a service swift] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:10:53.416227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb736c-8fcd-4855-aa2a-c97051d93e3a service swift] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:10:53.421527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9de56f-ca46-4bf3-b28d-d85fe5e543a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19779569e08a42148d2933f644e3b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:10:53.422637 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de56f-ca46-4bf3-b28d-d85fe5e543a2 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:10:53.422975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de56f-ca46-4bf3-b28d-d85fe5e543a2 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:10:53.423253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de56f-ca46-4bf3-b28d-d85fe5e543a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:10:53.431587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9de56f-ca46-4bf3-b28d-d85fe5e543a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:10:53.433157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9de56f-ca46-4bf3-b28d-d85fe5e543a2 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:10:53.442284 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1365/2743] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jul 28 04:10:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 04:11:03.377459 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:03.438996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:30.075979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5854fc1-f659-4893-8d10-e508f62a6b84 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:30.085552 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5854fc1-f659-4893-8d10-e508f62a6b84 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:30.089088 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c19e7-8860-43c9-8a63-f0c1184b51c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:30.089696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c19e7-8860-43c9-8a63-f0c1184b51c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:30.089985 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c19e7-8860-43c9-8a63-f0c1184b51c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:30.090263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c19e7-8860-43c9-8a63-f0c1184b51c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:30.092037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-599c19e7-8860-43c9-8a63-f0c1184b51c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:30.154621 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c19e7-8860-43c9-8a63-f0c1184b51c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:30.156076 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c19e7-8860-43c9-8a63-f0c1184b51c1 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:30.166219 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1379/2744] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:33.140425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9de56f-ca46-4bf3-b28d-d85fe5e543a2 service swift] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:33.142065 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3b9de56f-ca46-4bf3-b28d-d85fe5e543a2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:33.167937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9de56f-ca46-4bf3-b28d-d85fe5e543a2 service swift] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:33.181105 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca839105-8127-43e9-946e-8fa00277c9b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:33.181982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca839105-8127-43e9-946e-8fa00277c9b1 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:33.182487 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca839105-8127-43e9-946e-8fa00277c9b1 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:33.183038 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca839105-8127-43e9-946e-8fa00277c9b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:33.186657 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ca839105-8127-43e9-946e-8fa00277c9b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:33.203451 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca839105-8127-43e9-946e-8fa00277c9b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:33.206349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca839105-8127-43e9-946e-8fa00277c9b1 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:33.224806 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1366/2745] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:33.256527 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c19e7-8860-43c9-8a63-f0c1184b51c1 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:33.258393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-599c19e7-8860-43c9-8a63-f0c1184b51c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:33.273156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c19e7-8860-43c9-8a63-f0c1184b51c1 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:33.279657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c630075-3158-4c64-914e-581279572ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:33.280307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630075-3158-4c64-914e-581279572ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:33.280782 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630075-3158-4c64-914e-581279572ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:33.281040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630075-3158-4c64-914e-581279572ded admin admin] PATH_INFO: `/v3/users/b7e6ceb0f76d41ce962efd065ffa3d5a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:33.282209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c630075-3158-4c64-914e-581279572ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e6ceb0f76d41ce962efd065ffa3d5a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:33.284391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c630075-3158-4c64-914e-581279572ded admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:33.333647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c630075-3158-4c64-914e-581279572ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e6ceb0f76d41ce962efd065ffa3d5a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:33.334317 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c630075-3158-4c64-914e-581279572ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e6ceb0f76d41ce962efd065ffa3d5a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:33.345934 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c630075-3158-4c64-914e-581279572ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e6ceb0f76d41ce962efd065ffa3d5a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:33.354007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c630075-3158-4c64-914e-581279572ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e6ceb0f76d41ce962efd065ffa3d5a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:33.363129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c630075-3158-4c64-914e-581279572ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e6ceb0f76d41ce962efd065ffa3d5a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:33.370933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1380/2746] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:11:33 2022] DELETE /identity/v3/users/b7e6ceb0f76d41ce962efd065ffa3d5a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:33.548937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca839105-8127-43e9-946e-8fa00277c9b1 service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:33.565512 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca839105-8127-43e9-946e-8fa00277c9b1 service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:33.622611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:33.623272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:33.623548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:33.623817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] PATH_INFO: `/v3/projects/901cf93dff224196b6727a29f0038bdb` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:33.624954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] RBAC: Authorizing `identity:delete_project(project_id=901cf93dff224196b6727a29f0038bdb)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:33.626837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:33.672652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901cf93dff224196b6727a29f0038bdb', 'request_id': 'req-e328be3c-102c-4a40-9162-a44934c5b25f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:33.678046 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901cf93dff224196b6727a29f0038bdb', 'request_id': 'req-e328be3c-102c-4a40-9162-a44934c5b25f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:33.685149 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901cf93dff224196b6727a29f0038bdb', 'request_id': 'req-e328be3c-102c-4a40-9162-a44934c5b25f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:33.696761 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] The token cache is being invalidate because project 901cf93dff224196b6727a29f0038bdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:11:33.697053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:33.697691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e328be3c-102c-4a40-9162-a44934c5b25f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:33.699271 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1367/2747] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:11:33 2022] DELETE /identity/v3/projects/901cf93dff224196b6727a29f0038bdb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:33.704261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31476da9-5a60-4050-acff-de8a58263b03 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:33.704551 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31476da9-5a60-4050-acff-de8a58263b03 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:33.704833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31476da9-5a60-4050-acff-de8a58263b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:33.726989 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-31476da9-5a60-4050-acff-de8a58263b03 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:33.728191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-31476da9-5a60-4050-acff-de8a58263b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:33.789637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1381/2748] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:33.797590 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815aa4d4-9e41-450d-b214-27b7079466a5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:33.797881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815aa4d4-9e41-450d-b214-27b7079466a5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:33.798188 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815aa4d4-9e41-450d-b214-27b7079466a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:33.818511 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-815aa4d4-9e41-450d-b214-27b7079466a5 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:33.819752 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-815aa4d4-9e41-450d-b214-27b7079466a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:33.827765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1368/2749] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:11:33.831830 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31476da9-5a60-4050-acff-de8a58263b03 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:33.841466 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31476da9-5a60-4050-acff-de8a58263b03 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:33.846026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbf153-e488-416a-9903-91c7fecc9477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:33.846629 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbf153-e488-416a-9903-91c7fecc9477 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:33.846941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbf153-e488-416a-9903-91c7fecc9477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:33.847220 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbf153-e488-416a-9903-91c7fecc9477 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:33.847909 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbf153-e488-416a-9903-91c7fecc9477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:11:33.848369 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbf153-e488-416a-9903-91c7fecc9477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:33.849952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbf153-e488-416a-9903-91c7fecc9477 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:33.851421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:33.864762 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1382/2750] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:11:33.868847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815aa4d4-9e41-450d-b214-27b7079466a5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:33.878172 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815aa4d4-9e41-450d-b214-27b7079466a5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:33.883577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1290484-db40-430b-984d-e36944a259e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:33.884174 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1290484-db40-430b-984d-e36944a259e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:33.884459 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1290484-db40-430b-984d-e36944a259e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:33.884735 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1290484-db40-430b-984d-e36944a259e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:33.885401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1290484-db40-430b-984d-e36944a259e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:33.887430 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1290484-db40-430b-984d-e36944a259e4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:33.923233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1369/2751] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:11:33 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:11:33.928719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbf153-e488-416a-9903-91c7fecc9477 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:33.938505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbf153-e488-416a-9903-91c7fecc9477 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:33.994947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc771580-2d27-4e2b-9541-c145cd426279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:33.995561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc771580-2d27-4e2b-9541-c145cd426279 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:33.995868 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc771580-2d27-4e2b-9541-c145cd426279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:33.996140 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc771580-2d27-4e2b-9541-c145cd426279 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:33.996806 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc771580-2d27-4e2b-9541-c145cd426279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:33.998715 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc771580-2d27-4e2b-9541-c145cd426279 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:34.019658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1383/2752] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:11:33 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:11:34.023845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1290484-db40-430b-984d-e36944a259e4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:34.033990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1290484-db40-430b-984d-e36944a259e4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:34.038670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7b2d4-ec61-4f08-9c75-ba9e1fbe3803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:34.039567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7b2d4-ec61-4f08-9c75-ba9e1fbe3803 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:34.039845 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7b2d4-ec61-4f08-9c75-ba9e1fbe3803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:34.040117 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7b2d4-ec61-4f08-9c75-ba9e1fbe3803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:34.040807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7b2d4-ec61-4f08-9c75-ba9e1fbe3803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:34.042369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7b2d4-ec61-4f08-9c75-ba9e1fbe3803 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:34.052896 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1370/2753] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:34.056659 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc771580-2d27-4e2b-9541-c145cd426279 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:34.067649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc771580-2d27-4e2b-9541-c145cd426279 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:34.072285 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783167ac-2628-4f88-babd-dcc79e5dbe18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:34.072897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783167ac-2628-4f88-babd-dcc79e5dbe18 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:34.073171 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783167ac-2628-4f88-babd-dcc79e5dbe18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:34.073438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783167ac-2628-4f88-babd-dcc79e5dbe18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:34.074123 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783167ac-2628-4f88-babd-dcc79e5dbe18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:34.075730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783167ac-2628-4f88-babd-dcc79e5dbe18 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:34.086200 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1384/2754] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:34.090369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7b2d4-ec61-4f08-9c75-ba9e1fbe3803 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:34.100575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7b2d4-ec61-4f08-9c75-ba9e1fbe3803 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:34.105199 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6d123-21fd-46d6-a581-83cf486e99ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:34.105849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6d123-21fd-46d6-a581-83cf486e99ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:34.106133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6d123-21fd-46d6-a581-83cf486e99ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:34.106412 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6d123-21fd-46d6-a581-83cf486e99ab admin admin] PATH_INFO: `/v3/projects/e47e6e489fbc4019951175d2873a2f0f/users/54f8745e01ac41b6b69237d91375944f/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:34.121994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6d123-21fd-46d6-a581-83cf486e99ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=e47e6e489fbc4019951175d2873a2f0f, user_id=54f8745e01ac41b6b69237d91375944f, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:34.124738 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6d123-21fd-46d6-a581-83cf486e99ab admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:34.138131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1371/2755] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:34 2022] PUT /identity/v3/projects/e47e6e489fbc4019951175d2873a2f0f/users/54f8745e01ac41b6b69237d91375944f/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:34.143102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7ce4e-1ad7-490c-a7de-03f125dc156b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:34.143392 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7ce4e-1ad7-490c-a7de-03f125dc156b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:34.143791 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7ce4e-1ad7-490c-a7de-03f125dc156b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:34.163506 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a7d7ce4e-1ad7-490c-a7de-03f125dc156b None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:34.164964 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a7d7ce4e-1ad7-490c-a7de-03f125dc156b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:34.222397 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1385/2756] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:34.232467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6d123-21fd-46d6-a581-83cf486e99ab admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:34.234157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a2f6d123-21fd-46d6-a581-83cf486e99ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:34.241274 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:34.292158 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6d123-21fd-46d6-a581-83cf486e99ab admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:34.295927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b853356-140d-483f-985e-bbde44694650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:34.296541 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b853356-140d-483f-985e-bbde44694650 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:34.296820 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b853356-140d-483f-985e-bbde44694650 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:34.297091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b853356-140d-483f-985e-bbde44694650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:34.306968 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b853356-140d-483f-985e-bbde44694650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:34.308384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b853356-140d-483f-985e-bbde44694650 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:34.317647 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1372/2757] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:34.939400 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac2f26-d023-48c7-90f9-e8db1dfd8729 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:34.939952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac2f26-d023-48c7-90f9-e8db1dfd8729 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:34.940121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac2f26-d023-48c7-90f9-e8db1dfd8729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:34.969443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b3ac2f26-d023-48c7-90f9-e8db1dfd8729 None None] MFA Rules not processed for user `54f8745e01ac41b6b69237d91375944f`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:34.970760 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b3ac2f26-d023-48c7-90f9-e8db1dfd8729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:35.037263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1386/2758] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:35.042206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b853356-140d-483f-985e-bbde44694650 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:35.052596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b853356-140d-483f-985e-bbde44694650 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:35.057274 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce9386-67d7-477f-a236-c3a5a222c752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:35.057879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce9386-67d7-477f-a236-c3a5a222c752 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:35.058160 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce9386-67d7-477f-a236-c3a5a222c752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:35.058434 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce9386-67d7-477f-a236-c3a5a222c752 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:35.059139 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce9386-67d7-477f-a236-c3a5a222c752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:35.061021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce9386-67d7-477f-a236-c3a5a222c752 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:35.094121 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1373/2759] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:11:35 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:11:35.095119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac2f26-d023-48c7-90f9-e8db1dfd8729 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:35.100859 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:35.108313 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac2f26-d023-48c7-90f9-e8db1dfd8729 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:35.164873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba08415-4224-4f62-8d58-c2252b2ee326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:35.165487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba08415-4224-4f62-8d58-c2252b2ee326 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:35.165766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba08415-4224-4f62-8d58-c2252b2ee326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:35.166040 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba08415-4224-4f62-8d58-c2252b2ee326 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:35.166689 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba08415-4224-4f62-8d58-c2252b2ee326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:35.168637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba08415-4224-4f62-8d58-c2252b2ee326 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:35.188584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1387/2760] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:11:35 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:11:35.193461 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce9386-67d7-477f-a236-c3a5a222c752 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:35.203706 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce9386-67d7-477f-a236-c3a5a222c752 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:35.209102 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638833c5-2cad-4ca9-9ec8-360b5e0b1206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:35.209710 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638833c5-2cad-4ca9-9ec8-360b5e0b1206 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:35.210018 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638833c5-2cad-4ca9-9ec8-360b5e0b1206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:35.210294 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638833c5-2cad-4ca9-9ec8-360b5e0b1206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:35.211074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638833c5-2cad-4ca9-9ec8-360b5e0b1206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:35.212669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638833c5-2cad-4ca9-9ec8-360b5e0b1206 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:35.223060 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1374/2761] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:35.227241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba08415-4224-4f62-8d58-c2252b2ee326 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:35.237191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba08415-4224-4f62-8d58-c2252b2ee326 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:35.241755 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292c619-3aad-461c-bc02-d3ff692d50b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:35.242491 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292c619-3aad-461c-bc02-d3ff692d50b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:35.242776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292c619-3aad-461c-bc02-d3ff692d50b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:35.243078 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292c619-3aad-461c-bc02-d3ff692d50b3 admin admin] PATH_INFO: `/v3/projects/884e8321c906415dae83a0bcd5a49dfd/users/38fab234820f4ef6852bb2f981bed2ad/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:35.257931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292c619-3aad-461c-bc02-d3ff692d50b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=884e8321c906415dae83a0bcd5a49dfd, user_id=38fab234820f4ef6852bb2f981bed2ad, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:35.260988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292c619-3aad-461c-bc02-d3ff692d50b3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:35.274495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1388/2762] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:35 2022] PUT /identity/v3/projects/884e8321c906415dae83a0bcd5a49dfd/users/38fab234820f4ef6852bb2f981bed2ad/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:35.278669 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638833c5-2cad-4ca9-9ec8-360b5e0b1206 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:35.288680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638833c5-2cad-4ca9-9ec8-360b5e0b1206 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:35.343193 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed842e-21fc-4331-9daf-e264041984de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:35.343799 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed842e-21fc-4331-9daf-e264041984de admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:35.344074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed842e-21fc-4331-9daf-e264041984de admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:35.344401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed842e-21fc-4331-9daf-e264041984de admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:35.345019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed842e-21fc-4331-9daf-e264041984de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:35.346587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed842e-21fc-4331-9daf-e264041984de admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:35.356358 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1375/2763] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:35.360847 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292c619-3aad-461c-bc02-d3ff692d50b3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:35.371972 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292c619-3aad-461c-bc02-d3ff692d50b3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:35.376685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3578604-42b4-4e94-8e4c-59376403155f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:35.377337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3578604-42b4-4e94-8e4c-59376403155f admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:35.377620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3578604-42b4-4e94-8e4c-59376403155f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:35.377893 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3578604-42b4-4e94-8e4c-59376403155f admin admin] PATH_INFO: `/v3/projects/884e8321c906415dae83a0bcd5a49dfd/users/38fab234820f4ef6852bb2f981bed2ad/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:35.380516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3578604-42b4-4e94-8e4c-59376403155f admin admin] RBAC: Authorizing `identity:create_grant(project_id=884e8321c906415dae83a0bcd5a49dfd, user_id=38fab234820f4ef6852bb2f981bed2ad, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:35.383288 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3578604-42b4-4e94-8e4c-59376403155f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:35.394160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1389/2764] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:35 2022] PUT /identity/v3/projects/884e8321c906415dae83a0bcd5a49dfd/users/38fab234820f4ef6852bb2f981bed2ad/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:36.024112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138e21a-d6d0-4f9a-8181-fb913b826f52 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:36.024402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138e21a-d6d0-4f9a-8181-fb913b826f52 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:36.024676 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138e21a-d6d0-4f9a-8181-fb913b826f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:36.054692 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3138e21a-d6d0-4f9a-8181-fb913b826f52 None None] MFA Rules not processed for user `38fab234820f4ef6852bb2f981bed2ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:36.055936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3138e21a-d6d0-4f9a-8181-fb913b826f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:36.119394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:36.130831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1376/2765] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:36 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:36.587236 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3578604-42b4-4e94-8e4c-59376403155f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:36.589107 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a3578604-42b4-4e94-8e4c-59376403155f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:36.670248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3578604-42b4-4e94-8e4c-59376403155f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:36.675494 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc437ca4-3f56-4a26-98a8-eebb2b856c81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:36.676995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437ca4-3f56-4a26-98a8-eebb2b856c81 service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:36.677300 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437ca4-3f56-4a26-98a8-eebb2b856c81 service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:36.677577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437ca4-3f56-4a26-98a8-eebb2b856c81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:36.679437 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cc437ca4-3f56-4a26-98a8-eebb2b856c81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:36.691043 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc437ca4-3f56-4a26-98a8-eebb2b856c81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:36.692717 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc437ca4-3f56-4a26-98a8-eebb2b856c81 service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:36.702765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1390/2766] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:37.164230 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138e21a-d6d0-4f9a-8181-fb913b826f52 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:37.226765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138e21a-d6d0-4f9a-8181-fb913b826f52 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:37.231252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:37.231882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:37.232158 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:37.232431 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] PATH_INFO: `/v3/users/54f8745e01ac41b6b69237d91375944f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:37.233568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=54f8745e01ac41b6b69237d91375944f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:37.235520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:37.281860 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54f8745e01ac41b6b69237d91375944f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.282445 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54f8745e01ac41b6b69237d91375944f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.290403 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54f8745e01ac41b6b69237d91375944f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.302540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54f8745e01ac41b6b69237d91375944f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.309432 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54f8745e01ac41b6b69237d91375944f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.315359 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1377/2767] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:11:37 2022] DELETE /identity/v3/users/54f8745e01ac41b6b69237d91375944f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:37.488267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc437ca4-3f56-4a26-98a8-eebb2b856c81 service placement] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:37.500549 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc437ca4-3f56-4a26-98a8-eebb2b856c81 service placement] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:37.544075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:37.561329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:37.561981 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:37.562268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:37.562547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] PATH_INFO: `/v3/projects/e47e6e489fbc4019951175d2873a2f0f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:37.563779 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e47e6e489fbc4019951175d2873a2f0f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:37.565714 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:37.613377 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e47e6e489fbc4019951175d2873a2f0f', 'request_id': 'req-b5b05408-3a93-4c86-98d1-fd620215e9f2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.619328 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e47e6e489fbc4019951175d2873a2f0f', 'request_id': 'req-b5b05408-3a93-4c86-98d1-fd620215e9f2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.625497 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e47e6e489fbc4019951175d2873a2f0f', 'request_id': 'req-b5b05408-3a93-4c86-98d1-fd620215e9f2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.638018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] The token cache is being invalidate because project e47e6e489fbc4019951175d2873a2f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:11:37.638305 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.639008 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.640578 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1391/2768] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:11:37 2022] DELETE /identity/v3/projects/e47e6e489fbc4019951175d2873a2f0f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:37.644662 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:37.646091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:37.659875 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfec83-dfc4-428c-8bac-43d4bfca8ebd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:37.716310 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a64ef7-84f9-4186-a492-7f7e7b38db69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:37.716937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64ef7-84f9-4186-a492-7f7e7b38db69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:37.717217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64ef7-84f9-4186-a492-7f7e7b38db69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:37.717489 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64ef7-84f9-4186-a492-7f7e7b38db69 admin admin] PATH_INFO: `/v3/users/38fab234820f4ef6852bb2f981bed2ad` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:37.718639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a64ef7-84f9-4186-a492-7f7e7b38db69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38fab234820f4ef6852bb2f981bed2ad)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:37.720555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a64ef7-84f9-4186-a492-7f7e7b38db69 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:37.736653 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:37.765163 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f3a64ef7-84f9-4186-a492-7f7e7b38db69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38fab234820f4ef6852bb2f981bed2ad'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.765801 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f3a64ef7-84f9-4186-a492-7f7e7b38db69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38fab234820f4ef6852bb2f981bed2ad'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.772746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f3a64ef7-84f9-4186-a492-7f7e7b38db69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38fab234820f4ef6852bb2f981bed2ad'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.784279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f3a64ef7-84f9-4186-a492-7f7e7b38db69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38fab234820f4ef6852bb2f981bed2ad'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.791783 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f3a64ef7-84f9-4186-a492-7f7e7b38db69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38fab234820f4ef6852bb2f981bed2ad'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:37.797486 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1378/2769] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:11:37 2022] DELETE /identity/v3/users/38fab234820f4ef6852bb2f981bed2ad => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:37.977727 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:37.979215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:37.992596 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b05408-3a93-4c86-98d1-fd620215e9f2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:38.049430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:38.050098 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.050379 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.050648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] PATH_INFO: `/v3/projects/884e8321c906415dae83a0bcd5a49dfd` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.051816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=884e8321c906415dae83a0bcd5a49dfd)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:38.053845 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:38.100670 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884e8321c906415dae83a0bcd5a49dfd', 'request_id': 'req-446651dc-4adf-4f0c-ab22-f65baa119fe6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:38.106100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884e8321c906415dae83a0bcd5a49dfd', 'request_id': 'req-446651dc-4adf-4f0c-ab22-f65baa119fe6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:38.112365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884e8321c906415dae83a0bcd5a49dfd', 'request_id': 'req-446651dc-4adf-4f0c-ab22-f65baa119fe6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:38.125357 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] The token cache is being invalidate because project 884e8321c906415dae83a0bcd5a49dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:11:38.125645 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:38.126294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-446651dc-4adf-4f0c-ab22-f65baa119fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:38.127854 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1392/2770] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:11:37 2022] DELETE /identity/v3/projects/884e8321c906415dae83a0bcd5a49dfd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:11:38.133609 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3219b-f112-43e4-9e51-0222bb677f40 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.133893 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3219b-f112-43e4-9e51-0222bb677f40 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.134186 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3219b-f112-43e4-9e51-0222bb677f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.153893 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d4f3219b-f112-43e4-9e51-0222bb677f40 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:38.155302 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d4f3219b-f112-43e4-9e51-0222bb677f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:38.214171 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1379/2771] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:11:38.223572 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d79870-9f16-42ba-8088-46b5163961f6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.223857 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d79870-9f16-42ba-8088-46b5163961f6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.224128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d79870-9f16-42ba-8088-46b5163961f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.232738 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:38.245853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d3d79870-9f16-42ba-8088-46b5163961f6 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:38.247084 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d3d79870-9f16-42ba-8088-46b5163961f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:38.254872 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1393/2772] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:38.259832 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3219b-f112-43e4-9e51-0222bb677f40 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:38.269550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3219b-f112-43e4-9e51-0222bb677f40 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:38.274410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b83dcc-57c0-42ef-95d0-76d8935e6414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:38.275045 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b83dcc-57c0-42ef-95d0-76d8935e6414 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.275331 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b83dcc-57c0-42ef-95d0-76d8935e6414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.275605 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b83dcc-57c0-42ef-95d0-76d8935e6414 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.276349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b83dcc-57c0-42ef-95d0-76d8935e6414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:11:38.276654 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b83dcc-57c0-42ef-95d0-76d8935e6414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:38.278229 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b83dcc-57c0-42ef-95d0-76d8935e6414 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:38.290217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1380/2773] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:11:38.294331 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d79870-9f16-42ba-8088-46b5163961f6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:38.304136 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d79870-9f16-42ba-8088-46b5163961f6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:38.309390 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee15c42-cf08-4930-920f-c04a0758dee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:38.309988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee15c42-cf08-4930-920f-c04a0758dee8 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.310265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee15c42-cf08-4930-920f-c04a0758dee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.310537 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee15c42-cf08-4930-920f-c04a0758dee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.311239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee15c42-cf08-4930-920f-c04a0758dee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:38.313118 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee15c42-cf08-4930-920f-c04a0758dee8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:38.340554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1394/2774] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:11:38 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:38.344729 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b83dcc-57c0-42ef-95d0-76d8935e6414 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:38.355516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b83dcc-57c0-42ef-95d0-76d8935e6414 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:38.411241 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcf80f-fd2f-4e5d-9e59-0ca628f7977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:38.411847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf80f-fd2f-4e5d-9e59-0ca628f7977a admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.412130 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf80f-fd2f-4e5d-9e59-0ca628f7977a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.412405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf80f-fd2f-4e5d-9e59-0ca628f7977a admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.413232 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcf80f-fd2f-4e5d-9e59-0ca628f7977a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:38.415425 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcf80f-fd2f-4e5d-9e59-0ca628f7977a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:38.436261 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1381/2775] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:11:38 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:38.440315 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee15c42-cf08-4930-920f-c04a0758dee8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:38.451289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee15c42-cf08-4930-920f-c04a0758dee8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:38.455853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a44b2-1255-40c5-9293-e7ebac467fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:38.456455 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a44b2-1255-40c5-9293-e7ebac467fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.456735 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a44b2-1255-40c5-9293-e7ebac467fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.457005 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a44b2-1255-40c5-9293-e7ebac467fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.457692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a44b2-1255-40c5-9293-e7ebac467fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:38.459502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a44b2-1255-40c5-9293-e7ebac467fdb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:38.469531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1395/2776] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:38.473457 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcf80f-fd2f-4e5d-9e59-0ca628f7977a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:38.483501 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcf80f-fd2f-4e5d-9e59-0ca628f7977a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:38.488103 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fd3d1-6db9-4428-a843-107489b218f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:38.488699 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fd3d1-6db9-4428-a843-107489b218f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.488981 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fd3d1-6db9-4428-a843-107489b218f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.489254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fd3d1-6db9-4428-a843-107489b218f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.489944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fd3d1-6db9-4428-a843-107489b218f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:38.492217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fd3d1-6db9-4428-a843-107489b218f7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:38.497481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:38.503937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1382/2777] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:38.508873 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a44b2-1255-40c5-9293-e7ebac467fdb admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:38.518769 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a44b2-1255-40c5-9293-e7ebac467fdb admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:38.523616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63fbcf-204b-4e39-a32d-b21fa80d0cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:38.524263 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63fbcf-204b-4e39-a32d-b21fa80d0cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.524543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63fbcf-204b-4e39-a32d-b21fa80d0cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.524816 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63fbcf-204b-4e39-a32d-b21fa80d0cb7 admin admin] PATH_INFO: `/v3/projects/00f27ecf72c64235b944c9e2f71a26cc/users/cb868fef106c41949b25d30c19eb7e16/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.540191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63fbcf-204b-4e39-a32d-b21fa80d0cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f27ecf72c64235b944c9e2f71a26cc, user_id=cb868fef106c41949b25d30c19eb7e16, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:38.542855 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63fbcf-204b-4e39-a32d-b21fa80d0cb7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:38.556485 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1396/2778] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:38 2022] PUT /identity/v3/projects/00f27ecf72c64235b944c9e2f71a26cc/users/cb868fef106c41949b25d30c19eb7e16/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:38.619946 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdc9a1-4d9e-4768-a7d7-cc326c517d1c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.620230 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdc9a1-4d9e-4768-a7d7-cc326c517d1c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.620506 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdc9a1-4d9e-4768-a7d7-cc326c517d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.647697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d5fdc9a1-4d9e-4768-a7d7-cc326c517d1c None None] MFA Rules not processed for user `cb868fef106c41949b25d30c19eb7e16`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:38.648911 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d5fdc9a1-4d9e-4768-a7d7-cc326c517d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:38.711405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1383/2779] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:38.715866 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63fbcf-204b-4e39-a32d-b21fa80d0cb7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:38.776746 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63fbcf-204b-4e39-a32d-b21fa80d0cb7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:38.781329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ef110-dc45-4d1f-b8ec-6f009d946c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:38.781931 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef110-dc45-4d1f-b8ec-6f009d946c2c admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.782210 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef110-dc45-4d1f-b8ec-6f009d946c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.782487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef110-dc45-4d1f-b8ec-6f009d946c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.783176 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ef110-dc45-4d1f-b8ec-6f009d946c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:38.785051 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ef110-dc45-4d1f-b8ec-6f009d946c2c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:38.812983 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1397/2780] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:11:38 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:38.816997 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdc9a1-4d9e-4768-a7d7-cc326c517d1c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:38.827588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdc9a1-4d9e-4768-a7d7-cc326c517d1c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:38.883626 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df1a22-a1b7-414d-88ff-0f92d370e183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:38.884237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df1a22-a1b7-414d-88ff-0f92d370e183 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.884519 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df1a22-a1b7-414d-88ff-0f92d370e183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.884792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df1a22-a1b7-414d-88ff-0f92d370e183 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.885454 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df1a22-a1b7-414d-88ff-0f92d370e183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:38.887392 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df1a22-a1b7-414d-88ff-0f92d370e183 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:38.907479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1384/2781] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:11:38 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:11:38.911495 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ef110-dc45-4d1f-b8ec-6f009d946c2c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:38.922543 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ef110-dc45-4d1f-b8ec-6f009d946c2c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:38.927324 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634ce64-efc4-4b27-a91d-1a15609e64b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:38.927943 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634ce64-efc4-4b27-a91d-1a15609e64b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.928871 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634ce64-efc4-4b27-a91d-1a15609e64b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.929174 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634ce64-efc4-4b27-a91d-1a15609e64b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.929846 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634ce64-efc4-4b27-a91d-1a15609e64b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:38.931486 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634ce64-efc4-4b27-a91d-1a15609e64b0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:38.941938 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1398/2782] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:38.945898 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df1a22-a1b7-414d-88ff-0f92d370e183 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:38.955598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df1a22-a1b7-414d-88ff-0f92d370e183 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:38.961842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab9bfb-199f-4949-8ed4-e3214925ecd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:38.962540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab9bfb-199f-4949-8ed4-e3214925ecd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:38.962881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab9bfb-199f-4949-8ed4-e3214925ecd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:38.963127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab9bfb-199f-4949-8ed4-e3214925ecd0 admin admin] PATH_INFO: `/v3/projects/ffdc0cd3fec449c6a3977da076311bdc/users/01c4c3d0366d43278e4132f77d70f939/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:38.978711 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab9bfb-199f-4949-8ed4-e3214925ecd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffdc0cd3fec449c6a3977da076311bdc, user_id=01c4c3d0366d43278e4132f77d70f939, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:38.981410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab9bfb-199f-4949-8ed4-e3214925ecd0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:38.995217 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1385/2783] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:38 2022] PUT /identity/v3/projects/ffdc0cd3fec449c6a3977da076311bdc/users/01c4c3d0366d43278e4132f77d70f939/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:38.999275 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634ce64-efc4-4b27-a91d-1a15609e64b0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:39.009366 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634ce64-efc4-4b27-a91d-1a15609e64b0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:39.064343 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59608ca-3dca-484d-afa4-e384921d5698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:39.064883 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59608ca-3dca-484d-afa4-e384921d5698 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:39.065165 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59608ca-3dca-484d-afa4-e384921d5698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:39.065476 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59608ca-3dca-484d-afa4-e384921d5698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:39.066199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59608ca-3dca-484d-afa4-e384921d5698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:39.067813 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59608ca-3dca-484d-afa4-e384921d5698 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:39.078108 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1399/2784] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:39.082296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab9bfb-199f-4949-8ed4-e3214925ecd0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:39.092656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab9bfb-199f-4949-8ed4-e3214925ecd0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:39.097339 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a25fb-8e86-4447-9e5f-58ee8acb0b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:39.097990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a25fb-8e86-4447-9e5f-58ee8acb0b81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:39.098276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a25fb-8e86-4447-9e5f-58ee8acb0b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:39.098556 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a25fb-8e86-4447-9e5f-58ee8acb0b81 admin admin] PATH_INFO: `/v3/projects/ffdc0cd3fec449c6a3977da076311bdc/users/01c4c3d0366d43278e4132f77d70f939/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:39.100732 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a25fb-8e86-4447-9e5f-58ee8acb0b81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffdc0cd3fec449c6a3977da076311bdc, user_id=01c4c3d0366d43278e4132f77d70f939, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:39.103408 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a25fb-8e86-4447-9e5f-58ee8acb0b81 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:39.113499 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1386/2785] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:39 2022] PUT /identity/v3/projects/ffdc0cd3fec449c6a3977da076311bdc/users/01c4c3d0366d43278e4132f77d70f939/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:39.124624 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f186a4-60b2-4d3c-abc6-2f5fabb39811 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:39.124910 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f186a4-60b2-4d3c-abc6-2f5fabb39811 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:39.125185 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f186a4-60b2-4d3c-abc6-2f5fabb39811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:39.146146 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:39.154680 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e4f186a4-60b2-4d3c-abc6-2f5fabb39811 None None] MFA Rules not processed for user `01c4c3d0366d43278e4132f77d70f939`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:39.155928 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e4f186a4-60b2-4d3c-abc6-2f5fabb39811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:39.226278 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1400/2786] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:39.238026 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a25fb-8e86-4447-9e5f-58ee8acb0b81 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:39.239715 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c37a25fb-8e86-4447-9e5f-58ee8acb0b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:39.285221 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:39.298986 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a25fb-8e86-4447-9e5f-58ee8acb0b81 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:39.302697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bc060-31cc-476e-9fc5-560c84c1849f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:39.303950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bc060-31cc-476e-9fc5-560c84c1849f service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:39.304236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bc060-31cc-476e-9fc5-560c84c1849f service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:39.304516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bc060-31cc-476e-9fc5-560c84c1849f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:39.313779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bc060-31cc-476e-9fc5-560c84c1849f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:39.315299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bc060-31cc-476e-9fc5-560c84c1849f service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:39.325389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1387/2787] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:39.708658 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f186a4-60b2-4d3c-abc6-2f5fabb39811 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:39.769230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f186a4-60b2-4d3c-abc6-2f5fabb39811 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:39.774432 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:39.775104 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:39.775391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:39.775663 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] PATH_INFO: `/v3/users/cb868fef106c41949b25d30c19eb7e16` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:39.776841 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb868fef106c41949b25d30c19eb7e16)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:39.778759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:39.825415 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb868fef106c41949b25d30c19eb7e16'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:39.826044 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb868fef106c41949b25d30c19eb7e16'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:39.838587 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb868fef106c41949b25d30c19eb7e16'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:39.845462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb868fef106c41949b25d30c19eb7e16'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:39.852740 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb868fef106c41949b25d30c19eb7e16'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:39.858608 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1401/2788] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:11:39 2022] DELETE /identity/v3/users/cb868fef106c41949b25d30c19eb7e16 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:39.863260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771e40-4a1e-4dcb-9975-4c6efdff42b6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:39.863578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771e40-4a1e-4dcb-9975-4c6efdff42b6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:39.863927 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771e40-4a1e-4dcb-9975-4c6efdff42b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:39.884385 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b1771e40-4a1e-4dcb-9975-4c6efdff42b6 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:39.885607 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b1771e40-4a1e-4dcb-9975-4c6efdff42b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:39.943537 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1388/2789] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:39.953540 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:39.954969 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:39.973147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:40.013577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3d352-9858-491f-baad-4cf1a673fc55 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:40.017260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510efaf6-4510-4f8c-b2da-8405f974cec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:40.017870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510efaf6-4510-4f8c-b2da-8405f974cec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:40.018150 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510efaf6-4510-4f8c-b2da-8405f974cec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:40.018423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510efaf6-4510-4f8c-b2da-8405f974cec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:40.027483 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510efaf6-4510-4f8c-b2da-8405f974cec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:40.028915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510efaf6-4510-4f8c-b2da-8405f974cec8 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:40.038308 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1402/2790] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:40.306521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1771e40-4a1e-4dcb-9975-4c6efdff42b6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:40.308032 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b1771e40-4a1e-4dcb-9975-4c6efdff42b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:40.322308 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1771e40-4a1e-4dcb-9975-4c6efdff42b6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:40.327228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:40.327923 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:40.328207 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:40.328480 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] PATH_INFO: `/v3/projects/00f27ecf72c64235b944c9e2f71a26cc` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:40.329673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f27ecf72c64235b944c9e2f71a26cc)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:40.331708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:40.378210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f27ecf72c64235b944c9e2f71a26cc', 'request_id': 'req-6087773d-6d7c-41ab-8614-884f08139470'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.384466 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f27ecf72c64235b944c9e2f71a26cc', 'request_id': 'req-6087773d-6d7c-41ab-8614-884f08139470'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.396019 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f27ecf72c64235b944c9e2f71a26cc', 'request_id': 'req-6087773d-6d7c-41ab-8614-884f08139470'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.415843 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] The token cache is being invalidate because project 00f27ecf72c64235b944c9e2f71a26cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:11:40.416131 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.416764 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.418267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1389/2791] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:11:40 2022] DELETE /identity/v3/projects/00f27ecf72c64235b944c9e2f71a26cc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:40.422287 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510efaf6-4510-4f8c-b2da-8405f974cec8 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:40.431864 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510efaf6-4510-4f8c-b2da-8405f974cec8 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:40.488087 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bcbc4-fc1a-4294-8acb-0439c687a378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:40.488720 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bcbc4-fc1a-4294-8acb-0439c687a378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:40.488997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bcbc4-fc1a-4294-8acb-0439c687a378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:40.489266 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bcbc4-fc1a-4294-8acb-0439c687a378 admin admin] PATH_INFO: `/v3/users/01c4c3d0366d43278e4132f77d70f939` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:40.490404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bcbc4-fc1a-4294-8acb-0439c687a378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c4c3d0366d43278e4132f77d70f939)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:40.493037 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bcbc4-fc1a-4294-8acb-0439c687a378 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:40.538592 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-296bcbc4-fc1a-4294-8acb-0439c687a378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c4c3d0366d43278e4132f77d70f939'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.539556 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-296bcbc4-fc1a-4294-8acb-0439c687a378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c4c3d0366d43278e4132f77d70f939'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.551259 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-296bcbc4-fc1a-4294-8acb-0439c687a378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c4c3d0366d43278e4132f77d70f939'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.558684 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-296bcbc4-fc1a-4294-8acb-0439c687a378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c4c3d0366d43278e4132f77d70f939'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.567184 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-296bcbc4-fc1a-4294-8acb-0439c687a378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c4c3d0366d43278e4132f77d70f939'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.573940 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1403/2792] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:11:40 2022] DELETE /identity/v3/users/01c4c3d0366d43278e4132f77d70f939 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:40.846279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:40.847807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:40.863234 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087773d-6d7c-41ab-8614-884f08139470 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:40.921841 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:40.922407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:40.922691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:40.922998 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] PATH_INFO: `/v3/projects/ffdc0cd3fec449c6a3977da076311bdc` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:40.924146 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffdc0cd3fec449c6a3977da076311bdc)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:40.926050 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:40.936830 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:40.974670 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffdc0cd3fec449c6a3977da076311bdc', 'request_id': 'req-c9f07beb-0f18-4273-b8cc-734d88da7e0a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.980853 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffdc0cd3fec449c6a3977da076311bdc', 'request_id': 'req-c9f07beb-0f18-4273-b8cc-734d88da7e0a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.987542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffdc0cd3fec449c6a3977da076311bdc', 'request_id': 'req-c9f07beb-0f18-4273-b8cc-734d88da7e0a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:40.999227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] The token cache is being invalidate because project ffdc0cd3fec449c6a3977da076311bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:11:40.999517 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:41.000157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9f07beb-0f18-4273-b8cc-734d88da7e0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:41.001677 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1390/2793] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:11:40 2022] DELETE /identity/v3/projects/ffdc0cd3fec449c6a3977da076311bdc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:41.006599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c52129-1484-47cb-9a7b-1a3acec1a17a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.007079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c52129-1484-47cb-9a7b-1a3acec1a17a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.007190 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c52129-1484-47cb-9a7b-1a3acec1a17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.026199 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-45c52129-1484-47cb-9a7b-1a3acec1a17a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:41.027435 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-45c52129-1484-47cb-9a7b-1a3acec1a17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:41.060160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:41.090075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1404/2794] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:41.099125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62214dde-8d66-40f1-8eeb-d4f2248fa910 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.099421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62214dde-8d66-40f1-8eeb-d4f2248fa910 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.099703 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62214dde-8d66-40f1-8eeb-d4f2248fa910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.119030 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-62214dde-8d66-40f1-8eeb-d4f2248fa910 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:41.120239 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-62214dde-8d66-40f1-8eeb-d4f2248fa910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:41.128120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1391/2795] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:41.132752 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c52129-1484-47cb-9a7b-1a3acec1a17a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.142249 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c52129-1484-47cb-9a7b-1a3acec1a17a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.146759 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263662f-951f-4ecd-a445-03d8b6301d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.147914 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263662f-951f-4ecd-a445-03d8b6301d3e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.148196 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263662f-951f-4ecd-a445-03d8b6301d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.148467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263662f-951f-4ecd-a445-03d8b6301d3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.149151 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263662f-951f-4ecd-a445-03d8b6301d3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:11:41.149452 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263662f-951f-4ecd-a445-03d8b6301d3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.151061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263662f-951f-4ecd-a445-03d8b6301d3e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.162475 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1405/2796] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:11:41.166904 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62214dde-8d66-40f1-8eeb-d4f2248fa910 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.176101 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62214dde-8d66-40f1-8eeb-d4f2248fa910 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.180932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc856a83-95d4-42c3-8836-1b934f1480bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.181529 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc856a83-95d4-42c3-8836-1b934f1480bb admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.181819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc856a83-95d4-42c3-8836-1b934f1480bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.182091 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc856a83-95d4-42c3-8836-1b934f1480bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.182740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc856a83-95d4-42c3-8836-1b934f1480bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.184707 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc856a83-95d4-42c3-8836-1b934f1480bb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.211951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1392/2797] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:11:41 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:41.216067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263662f-951f-4ecd-a445-03d8b6301d3e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.225922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263662f-951f-4ecd-a445-03d8b6301d3e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.281947 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5dc8a-d879-44b7-8b07-8ee007533326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.282616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5dc8a-d879-44b7-8b07-8ee007533326 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.282937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5dc8a-d879-44b7-8b07-8ee007533326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.283241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5dc8a-d879-44b7-8b07-8ee007533326 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.283915 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5dc8a-d879-44b7-8b07-8ee007533326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.285832 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5dc8a-d879-44b7-8b07-8ee007533326 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.308446 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1406/2798] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:11:41 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:41.312520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc856a83-95d4-42c3-8836-1b934f1480bb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.323583 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc856a83-95d4-42c3-8836-1b934f1480bb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.328210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a171cd-f016-428d-a7dc-808bfe056977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.328815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a171cd-f016-428d-a7dc-808bfe056977 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.329098 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a171cd-f016-428d-a7dc-808bfe056977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.329373 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a171cd-f016-428d-a7dc-808bfe056977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.330069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a171cd-f016-428d-a7dc-808bfe056977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.331680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a171cd-f016-428d-a7dc-808bfe056977 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.342118 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1393/2799] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 04:11:41.345911 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5dc8a-d879-44b7-8b07-8ee007533326 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.356924 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5dc8a-d879-44b7-8b07-8ee007533326 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.361649 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcba327-afec-4f56-b2cf-76c27ab4ed20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.362255 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba327-afec-4f56-b2cf-76c27ab4ed20 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.362539 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba327-afec-4f56-b2cf-76c27ab4ed20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.362843 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba327-afec-4f56-b2cf-76c27ab4ed20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.363569 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcba327-afec-4f56-b2cf-76c27ab4ed20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.365154 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcba327-afec-4f56-b2cf-76c27ab4ed20 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.376036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1407/2800] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:41.380479 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a171cd-f016-428d-a7dc-808bfe056977 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.391366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a171cd-f016-428d-a7dc-808bfe056977 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.395915 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b11d1-9408-4fb5-875e-a1c911b835e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.396568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b11d1-9408-4fb5-875e-a1c911b835e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.397397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b11d1-9408-4fb5-875e-a1c911b835e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.397780 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b11d1-9408-4fb5-875e-a1c911b835e8 admin admin] PATH_INFO: `/v3/projects/8a10be487ae54635a772bcc83c13f02a/users/1f0eec36eb074751b7fa84d9bbdb7688/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.411993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b11d1-9408-4fb5-875e-a1c911b835e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a10be487ae54635a772bcc83c13f02a, user_id=1f0eec36eb074751b7fa84d9bbdb7688, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.415795 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b11d1-9408-4fb5-875e-a1c911b835e8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.428831 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1394/2801] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:41 2022] PUT /identity/v3/projects/8a10be487ae54635a772bcc83c13f02a/users/1f0eec36eb074751b7fa84d9bbdb7688/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:41.438948 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df8275-ced1-42e2-bb60-28ae07500fe0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.439246 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df8275-ced1-42e2-bb60-28ae07500fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.439528 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df8275-ced1-42e2-bb60-28ae07500fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.467247 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-49df8275-ced1-42e2-bb60-28ae07500fe0 None None] MFA Rules not processed for user `1f0eec36eb074751b7fa84d9bbdb7688`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:41.468463 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-49df8275-ced1-42e2-bb60-28ae07500fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:41.531803 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1408/2802] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:41.536361 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b11d1-9408-4fb5-875e-a1c911b835e8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.598210 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b11d1-9408-4fb5-875e-a1c911b835e8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.603661 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e491e-0ceb-4bc3-a76e-0dfb18da542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.604267 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e491e-0ceb-4bc3-a76e-0dfb18da542f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.604549 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e491e-0ceb-4bc3-a76e-0dfb18da542f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.604825 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e491e-0ceb-4bc3-a76e-0dfb18da542f admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.605491 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e491e-0ceb-4bc3-a76e-0dfb18da542f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.607534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e491e-0ceb-4bc3-a76e-0dfb18da542f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.633920 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1395/2803] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:11:41 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:41.637951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df8275-ced1-42e2-bb60-28ae07500fe0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.649736 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df8275-ced1-42e2-bb60-28ae07500fe0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.680568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:41.707671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283e4e1-2d9c-4fe7-9ce5-059dded90081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.708289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283e4e1-2d9c-4fe7-9ce5-059dded90081 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.708576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283e4e1-2d9c-4fe7-9ce5-059dded90081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.708858 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283e4e1-2d9c-4fe7-9ce5-059dded90081 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.710085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283e4e1-2d9c-4fe7-9ce5-059dded90081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.712038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283e4e1-2d9c-4fe7-9ce5-059dded90081 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.732733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1409/2804] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:11:41 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:41.736834 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e491e-0ceb-4bc3-a76e-0dfb18da542f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.747293 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e491e-0ceb-4bc3-a76e-0dfb18da542f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.751852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b139d-5693-4309-bafc-bb388a839c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.752474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b139d-5693-4309-bafc-bb388a839c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.752759 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b139d-5693-4309-bafc-bb388a839c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.753031 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b139d-5693-4309-bafc-bb388a839c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.753713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b139d-5693-4309-bafc-bb388a839c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.755367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b139d-5693-4309-bafc-bb388a839c02 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.766177 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1396/2805] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:41.770265 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283e4e1-2d9c-4fe7-9ce5-059dded90081 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.781042 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283e4e1-2d9c-4fe7-9ce5-059dded90081 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.785992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc50bca-3233-4064-baa6-9a2907ef8c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.786653 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc50bca-3233-4064-baa6-9a2907ef8c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.787001 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc50bca-3233-4064-baa6-9a2907ef8c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.787421 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc50bca-3233-4064-baa6-9a2907ef8c02 admin admin] PATH_INFO: `/v3/projects/c5f34fe9550440dfac455cbef03bafed/users/347da1fdc17a4dddb26d6634c1e156db/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.802275 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc50bca-3233-4064-baa6-9a2907ef8c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f34fe9550440dfac455cbef03bafed, user_id=347da1fdc17a4dddb26d6634c1e156db, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.805365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc50bca-3233-4064-baa6-9a2907ef8c02 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.817842 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1410/2806] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:41 2022] PUT /identity/v3/projects/c5f34fe9550440dfac455cbef03bafed/users/347da1fdc17a4dddb26d6634c1e156db/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:41.822539 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b139d-5693-4309-bafc-bb388a839c02 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.832395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b139d-5693-4309-bafc-bb388a839c02 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.837958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:41.892014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f54e81-2c2e-4b8d-9a08-bd24e96dbc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.892633 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f54e81-2c2e-4b8d-9a08-bd24e96dbc11 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.892916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f54e81-2c2e-4b8d-9a08-bd24e96dbc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.893192 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f54e81-2c2e-4b8d-9a08-bd24e96dbc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.893906 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f54e81-2c2e-4b8d-9a08-bd24e96dbc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.895516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f54e81-2c2e-4b8d-9a08-bd24e96dbc11 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.906006 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1397/2807] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:41.910293 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc50bca-3233-4064-baa6-9a2907ef8c02 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:41.921020 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc50bca-3233-4064-baa6-9a2907ef8c02 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:41.925700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2434e-6c3e-40ef-b1a8-c3f7a5d852bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:41.926358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2434e-6c3e-40ef-b1a8-c3f7a5d852bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.926644 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2434e-6c3e-40ef-b1a8-c3f7a5d852bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.926967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2434e-6c3e-40ef-b1a8-c3f7a5d852bb admin admin] PATH_INFO: `/v3/projects/c5f34fe9550440dfac455cbef03bafed/users/347da1fdc17a4dddb26d6634c1e156db/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.929181 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2434e-6c3e-40ef-b1a8-c3f7a5d852bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f34fe9550440dfac455cbef03bafed, user_id=347da1fdc17a4dddb26d6634c1e156db, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:41.931902 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2434e-6c3e-40ef-b1a8-c3f7a5d852bb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:41.941834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1411/2808] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:41 2022] PUT /identity/v3/projects/c5f34fe9550440dfac455cbef03bafed/users/347da1fdc17a4dddb26d6634c1e156db/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:11:41.952783 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c811b0-ad5b-4a9a-8c68-0a9371e71914 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:41.953075 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c811b0-ad5b-4a9a-8c68-0a9371e71914 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:41.953353 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c811b0-ad5b-4a9a-8c68-0a9371e71914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:41.981713 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-14c811b0-ad5b-4a9a-8c68-0a9371e71914 None None] MFA Rules not processed for user `347da1fdc17a4dddb26d6634c1e156db`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:41.983012 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-14c811b0-ad5b-4a9a-8c68-0a9371e71914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:42.054007 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1398/2809] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:41 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:42.064798 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2434e-6c3e-40ef-b1a8-c3f7a5d852bb admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:42.066608 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-49b2434e-6c3e-40ef-b1a8-c3f7a5d852bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:42.127066 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2434e-6c3e-40ef-b1a8-c3f7a5d852bb admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:42.130728 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dccc32-17d8-45bb-85d3-8fbc11312e14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:42.131829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dccc32-17d8-45bb-85d3-8fbc11312e14 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:42.132119 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dccc32-17d8-45bb-85d3-8fbc11312e14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:42.132398 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dccc32-17d8-45bb-85d3-8fbc11312e14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:42.141299 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dccc32-17d8-45bb-85d3-8fbc11312e14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:42.142718 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dccc32-17d8-45bb-85d3-8fbc11312e14 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:42.152916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1412/2810] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:42 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:43.485410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c811b0-ad5b-4a9a-8c68-0a9371e71914 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:43.548039 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c811b0-ad5b-4a9a-8c68-0a9371e71914 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:43.552652 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:43.553315 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:43.553593 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:43.553885 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] PATH_INFO: `/v3/users/1f0eec36eb074751b7fa84d9bbdb7688` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:43.555288 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f0eec36eb074751b7fa84d9bbdb7688)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:43.557212 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:43.590847 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:43.605154 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f0eec36eb074751b7fa84d9bbdb7688'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:43.605854 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f0eec36eb074751b7fa84d9bbdb7688'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:43.612671 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f0eec36eb074751b7fa84d9bbdb7688'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:43.624452 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f0eec36eb074751b7fa84d9bbdb7688'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:43.631319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f0eec36eb074751b7fa84d9bbdb7688'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:43.637204 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1399/2811] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:11:43 2022] DELETE /identity/v3/users/1f0eec36eb074751b7fa84d9bbdb7688 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:43.641673 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b78b02-a472-464c-864b-076285fa9bf4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:43.642143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b78b02-a472-464c-864b-076285fa9bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:43.642241 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b78b02-a472-464c-864b-076285fa9bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:43.661195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-18b78b02-a472-464c-864b-076285fa9bf4 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:43.662399 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-18b78b02-a472-464c-864b-076285fa9bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:43.720616 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1413/2812] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:43.730138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:43.731619 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:43.787842 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc02d3-5e21-4458-8a90-d081228aa32b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:43.791792 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f64b3-0bf9-4859-ba7f-c8c6af78c945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:43.792405 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f64b3-0bf9-4859-ba7f-c8c6af78c945 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:43.792683 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f64b3-0bf9-4859-ba7f-c8c6af78c945 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:43.792953 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f64b3-0bf9-4859-ba7f-c8c6af78c945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:43.801715 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f64b3-0bf9-4859-ba7f-c8c6af78c945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:43.803166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f64b3-0bf9-4859-ba7f-c8c6af78c945 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:43.813144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1400/2813] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:44.086962 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b78b02-a472-464c-864b-076285fa9bf4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:44.088500 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-18b78b02-a472-464c-864b-076285fa9bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:44.102061 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b78b02-a472-464c-864b-076285fa9bf4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:44.107467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:44.108141 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:44.108387 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:44.108672 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] PATH_INFO: `/v3/projects/8a10be487ae54635a772bcc83c13f02a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:44.109812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a10be487ae54635a772bcc83c13f02a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:44.111758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:44.137137 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:44.161068 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a10be487ae54635a772bcc83c13f02a', 'request_id': 'req-dbd9201a-4562-4118-97b4-89863480098b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.166941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a10be487ae54635a772bcc83c13f02a', 'request_id': 'req-dbd9201a-4562-4118-97b4-89863480098b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.173447 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a10be487ae54635a772bcc83c13f02a', 'request_id': 'req-dbd9201a-4562-4118-97b4-89863480098b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.186631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] The token cache is being invalidate because project 8a10be487ae54635a772bcc83c13f02a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:11:44.186951 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.187577 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.189085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1414/2814] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:11:44 2022] DELETE /identity/v3/projects/8a10be487ae54635a772bcc83c13f02a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:11:44.193112 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f64b3-0bf9-4859-ba7f-c8c6af78c945 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:44.203536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f64b3-0bf9-4859-ba7f-c8c6af78c945 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:44.259740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f286196-3901-43e5-83f8-1fec7afb51ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:44.260303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f286196-3901-43e5-83f8-1fec7afb51ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:44.260580 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f286196-3901-43e5-83f8-1fec7afb51ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:44.260861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f286196-3901-43e5-83f8-1fec7afb51ed admin admin] PATH_INFO: `/v3/users/347da1fdc17a4dddb26d6634c1e156db` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:44.262025 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f286196-3901-43e5-83f8-1fec7afb51ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=347da1fdc17a4dddb26d6634c1e156db)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:44.263996 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f286196-3901-43e5-83f8-1fec7afb51ed admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:44.314069 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1f286196-3901-43e5-83f8-1fec7afb51ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '347da1fdc17a4dddb26d6634c1e156db'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.314726 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1f286196-3901-43e5-83f8-1fec7afb51ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '347da1fdc17a4dddb26d6634c1e156db'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.322327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1f286196-3901-43e5-83f8-1fec7afb51ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '347da1fdc17a4dddb26d6634c1e156db'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.323621 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:44.337010 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1f286196-3901-43e5-83f8-1fec7afb51ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '347da1fdc17a4dddb26d6634c1e156db'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.343727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1f286196-3901-43e5-83f8-1fec7afb51ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '347da1fdc17a4dddb26d6634c1e156db'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.349549 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1401/2815] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:11:44 2022] DELETE /identity/v3/users/347da1fdc17a4dddb26d6634c1e156db => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:44.621372 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:44.622940 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:44.638070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9201a-4562-4118-97b4-89863480098b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:44.697690 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:44.698321 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:44.698591 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:44.698896 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] PATH_INFO: `/v3/projects/c5f34fe9550440dfac455cbef03bafed` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:44.700048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f34fe9550440dfac455cbef03bafed)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:44.701936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:44.747801 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f34fe9550440dfac455cbef03bafed', 'request_id': 'req-29228630-0a39-4f38-b86f-50ac090ff161'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.753213 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f34fe9550440dfac455cbef03bafed', 'request_id': 'req-29228630-0a39-4f38-b86f-50ac090ff161'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.759230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f34fe9550440dfac455cbef03bafed', 'request_id': 'req-29228630-0a39-4f38-b86f-50ac090ff161'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.771156 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] The token cache is being invalidate because project c5f34fe9550440dfac455cbef03bafed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:11:44.771401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.772102 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-29228630-0a39-4f38-b86f-50ac090ff161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:44.773851 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1415/2816] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:11:44 2022] DELETE /identity/v3/projects/c5f34fe9550440dfac455cbef03bafed => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:44.778659 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51ad95-b044-47e7-88c6-bc9c3e53d3c1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:44.779134 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51ad95-b044-47e7-88c6-bc9c3e53d3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:44.779252 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51ad95-b044-47e7-88c6-bc9c3e53d3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:44.799253 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ba51ad95-b044-47e7-88c6-bc9c3e53d3c1 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:44.800467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ba51ad95-b044-47e7-88c6-bc9c3e53d3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:44.858218 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1402/2817] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:44.865868 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d438c03-4dc1-4372-b414-3496506455b3 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:44.866163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d438c03-4dc1-4372-b414-3496506455b3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:44.867038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d438c03-4dc1-4372-b414-3496506455b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:44.886692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5d438c03-4dc1-4372-b414-3496506455b3 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:44.887926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5d438c03-4dc1-4372-b414-3496506455b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:44.895859 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1416/2818] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:44.900037 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51ad95-b044-47e7-88c6-bc9c3e53d3c1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:44.909857 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51ad95-b044-47e7-88c6-bc9c3e53d3c1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:44.915227 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89182722-4053-4813-8bf3-5e7d39b0eda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:44.915828 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89182722-4053-4813-8bf3-5e7d39b0eda5 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:44.916107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89182722-4053-4813-8bf3-5e7d39b0eda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:44.916384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89182722-4053-4813-8bf3-5e7d39b0eda5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:44.917071 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89182722-4053-4813-8bf3-5e7d39b0eda5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:11:44.917370 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89182722-4053-4813-8bf3-5e7d39b0eda5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:44.918975 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89182722-4053-4813-8bf3-5e7d39b0eda5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:44.931008 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1403/2819] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:11:44.935153 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d438c03-4dc1-4372-b414-3496506455b3 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:44.944312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d438c03-4dc1-4372-b414-3496506455b3 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:44.949544 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1123859-c92f-45f5-b2f4-f378df869452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:44.950140 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1123859-c92f-45f5-b2f4-f378df869452 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:44.950423 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1123859-c92f-45f5-b2f4-f378df869452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:44.950697 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1123859-c92f-45f5-b2f4-f378df869452 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:44.951413 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1123859-c92f-45f5-b2f4-f378df869452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:44.953299 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1123859-c92f-45f5-b2f4-f378df869452 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:44.981469 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1417/2820] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:11:44 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:44.985528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89182722-4053-4813-8bf3-5e7d39b0eda5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:44.995844 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89182722-4053-4813-8bf3-5e7d39b0eda5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.051215 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcdb3a-7568-42dd-84a2-dcf0465151df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.051811 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcdb3a-7568-42dd-84a2-dcf0465151df admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.052094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcdb3a-7568-42dd-84a2-dcf0465151df admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.052367 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcdb3a-7568-42dd-84a2-dcf0465151df admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.053017 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcdb3a-7568-42dd-84a2-dcf0465151df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:45.055092 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcdb3a-7568-42dd-84a2-dcf0465151df admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:45.075542 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1404/2821] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:11:44 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:45.079825 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1123859-c92f-45f5-b2f4-f378df869452 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:45.090730 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1123859-c92f-45f5-b2f4-f378df869452 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.095329 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc251648-bdca-4200-a1a7-95c22a349623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.095933 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc251648-bdca-4200-a1a7-95c22a349623 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.096215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc251648-bdca-4200-a1a7-95c22a349623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.096492 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc251648-bdca-4200-a1a7-95c22a349623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.097191 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc251648-bdca-4200-a1a7-95c22a349623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:45.098791 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc251648-bdca-4200-a1a7-95c22a349623 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:45.109122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1418/2822] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:45.113090 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcdb3a-7568-42dd-84a2-dcf0465151df admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:45.123166 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcdb3a-7568-42dd-84a2-dcf0465151df admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.127810 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c7d73-836f-4630-94c5-2a0d4fb0327d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.128410 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c7d73-836f-4630-94c5-2a0d4fb0327d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.128696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c7d73-836f-4630-94c5-2a0d4fb0327d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.128974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c7d73-836f-4630-94c5-2a0d4fb0327d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.129666 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c7d73-836f-4630-94c5-2a0d4fb0327d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:45.131299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c7d73-836f-4630-94c5-2a0d4fb0327d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:45.141730 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1405/2823] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:45.145747 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc251648-bdca-4200-a1a7-95c22a349623 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:45.151471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:45.157807 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc251648-bdca-4200-a1a7-95c22a349623 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.164260 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96294cf4-c10b-4cdb-ae28-588993d16d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.164918 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294cf4-c10b-4cdb-ae28-588993d16d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.165203 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294cf4-c10b-4cdb-ae28-588993d16d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.165481 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294cf4-c10b-4cdb-ae28-588993d16d87 admin admin] PATH_INFO: `/v3/projects/d1c0ddb6ad9b4dc39d7452c57b70faa4/users/0b5e2ca5924c4741bab9dd4728bf14c8/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.180018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96294cf4-c10b-4cdb-ae28-588993d16d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c0ddb6ad9b4dc39d7452c57b70faa4, user_id=0b5e2ca5924c4741bab9dd4728bf14c8, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:45.182651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96294cf4-c10b-4cdb-ae28-588993d16d87 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:45.195055 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1419/2824] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:45 2022] PUT /identity/v3/projects/d1c0ddb6ad9b4dc39d7452c57b70faa4/users/0b5e2ca5924c4741bab9dd4728bf14c8/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:11:45.205356 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6b4c1-f17e-4bd0-9598-8b057b6451c1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.205646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6b4c1-f17e-4bd0-9598-8b057b6451c1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.205921 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6b4c1-f17e-4bd0-9598-8b057b6451c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.233740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6da6b4c1-f17e-4bd0-9598-8b057b6451c1 None None] MFA Rules not processed for user `0b5e2ca5924c4741bab9dd4728bf14c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:45.234978 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6da6b4c1-f17e-4bd0-9598-8b057b6451c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:45.260314 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:45.300437 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1406/2825] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:45.305179 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96294cf4-c10b-4cdb-ae28-588993d16d87 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:45.367130 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96294cf4-c10b-4cdb-ae28-588993d16d87 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.371876 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de95e32-808a-4c16-86b3-bb05a468d206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.372613 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de95e32-808a-4c16-86b3-bb05a468d206 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.372897 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de95e32-808a-4c16-86b3-bb05a468d206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.373172 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de95e32-808a-4c16-86b3-bb05a468d206 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.373844 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de95e32-808a-4c16-86b3-bb05a468d206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:45.375782 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de95e32-808a-4c16-86b3-bb05a468d206 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:45.402661 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1420/2826] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:11:45 2022] POST /identity/v3/projects => generated 391 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:11:45.406740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6b4c1-f17e-4bd0-9598-8b057b6451c1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:45.418401 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6b4c1-f17e-4bd0-9598-8b057b6451c1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.475577 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7f30a-a854-44ae-adb4-036c925a33cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.476889 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7f30a-a854-44ae-adb4-036c925a33cb admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.477170 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7f30a-a854-44ae-adb4-036c925a33cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.477472 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7f30a-a854-44ae-adb4-036c925a33cb admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.478127 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7f30a-a854-44ae-adb4-036c925a33cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:45.480068 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7f30a-a854-44ae-adb4-036c925a33cb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:45.500582 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1407/2827] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:11:45 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:45.504572 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de95e32-808a-4c16-86b3-bb05a468d206 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:45.515303 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de95e32-808a-4c16-86b3-bb05a468d206 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.519870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df371b4-3528-416d-8ae3-4bd64443bd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.520478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df371b4-3528-416d-8ae3-4bd64443bd2d admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.520756 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df371b4-3528-416d-8ae3-4bd64443bd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.521030 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df371b4-3528-416d-8ae3-4bd64443bd2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.521719 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df371b4-3528-416d-8ae3-4bd64443bd2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:45.523571 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df371b4-3528-416d-8ae3-4bd64443bd2d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:45.534079 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1421/2828] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:45.538737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7f30a-a854-44ae-adb4-036c925a33cb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:45.548990 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7f30a-a854-44ae-adb4-036c925a33cb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.554482 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bcaef-80f1-4d15-a346-a89414438256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.555247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bcaef-80f1-4d15-a346-a89414438256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.555471 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bcaef-80f1-4d15-a346-a89414438256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.555740 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bcaef-80f1-4d15-a346-a89414438256 admin admin] PATH_INFO: `/v3/projects/1cba8350deac4616a68a40b5ea2ae740/users/b7e4c01f931c46acbbdc1963987daebe/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.570419 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bcaef-80f1-4d15-a346-a89414438256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cba8350deac4616a68a40b5ea2ae740, user_id=b7e4c01f931c46acbbdc1963987daebe, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:45.573278 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bcaef-80f1-4d15-a346-a89414438256 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:45.585839 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1408/2829] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:45 2022] PUT /identity/v3/projects/1cba8350deac4616a68a40b5ea2ae740/users/b7e4c01f931c46acbbdc1963987daebe/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:45.589553 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df371b4-3528-416d-8ae3-4bd64443bd2d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:45.599467 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df371b4-3528-416d-8ae3-4bd64443bd2d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.656048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94251a62-0486-4c6e-ad6d-f7709ffb1b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.656652 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94251a62-0486-4c6e-ad6d-f7709ffb1b52 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.656936 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94251a62-0486-4c6e-ad6d-f7709ffb1b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.657212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94251a62-0486-4c6e-ad6d-f7709ffb1b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.657899 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94251a62-0486-4c6e-ad6d-f7709ffb1b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:45.659530 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94251a62-0486-4c6e-ad6d-f7709ffb1b52 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:45.669942 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1422/2830] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:45.674395 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bcaef-80f1-4d15-a346-a89414438256 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:45.685446 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bcaef-80f1-4d15-a346-a89414438256 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.690048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ac2fe-72d7-47c3-be9f-23aac0e53436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.690700 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ac2fe-72d7-47c3-be9f-23aac0e53436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.691013 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ac2fe-72d7-47c3-be9f-23aac0e53436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.691291 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ac2fe-72d7-47c3-be9f-23aac0e53436 admin admin] PATH_INFO: `/v3/projects/1cba8350deac4616a68a40b5ea2ae740/users/b7e4c01f931c46acbbdc1963987daebe/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.693219 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ac2fe-72d7-47c3-be9f-23aac0e53436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cba8350deac4616a68a40b5ea2ae740, user_id=b7e4c01f931c46acbbdc1963987daebe, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:45.695901 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ac2fe-72d7-47c3-be9f-23aac0e53436 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:45.705887 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1409/2831] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:45 2022] PUT /identity/v3/projects/1cba8350deac4616a68a40b5ea2ae740/users/b7e4c01f931c46acbbdc1963987daebe/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:45.717642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe903c52-918b-4797-852b-64dfdc96e11d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.718135 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe903c52-918b-4797-852b-64dfdc96e11d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.718248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe903c52-918b-4797-852b-64dfdc96e11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.746695 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fe903c52-918b-4797-852b-64dfdc96e11d None None] MFA Rules not processed for user `b7e4c01f931c46acbbdc1963987daebe`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:45.747926 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fe903c52-918b-4797-852b-64dfdc96e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:45.785765 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:45.816776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1423/2832] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:45.827954 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ac2fe-72d7-47c3-be9f-23aac0e53436 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:45.829763 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2b4ac2fe-72d7-47c3-be9f-23aac0e53436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:45.887869 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ac2fe-72d7-47c3-be9f-23aac0e53436 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.891544 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c069-145b-48a7-aa0d-97803e517533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.892153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c069-145b-48a7-aa0d-97803e517533 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.892439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c069-145b-48a7-aa0d-97803e517533 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.892715 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c069-145b-48a7-aa0d-97803e517533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:45.902340 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8c069-145b-48a7-aa0d-97803e517533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:45.903794 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8c069-145b-48a7-aa0d-97803e517533 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:45.913236 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1410/2833] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:45.960027 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe903c52-918b-4797-852b-64dfdc96e11d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:45.961771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fe903c52-918b-4797-852b-64dfdc96e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:45.973667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe903c52-918b-4797-852b-64dfdc96e11d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:45.977338 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bacc62-3614-411f-a97a-41b812210259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:45.977937 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bacc62-3614-411f-a97a-41b812210259 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:45.978222 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bacc62-3614-411f-a97a-41b812210259 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:45.978493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bacc62-3614-411f-a97a-41b812210259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:46.033389 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bacc62-3614-411f-a97a-41b812210259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:46.034863 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bacc62-3614-411f-a97a-41b812210259 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:46.043997 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1424/2834] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:49.296357 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c069-145b-48a7-aa0d-97803e517533 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:49.325716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:49.363094 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c069-145b-48a7-aa0d-97803e517533 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:49.367596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:49.368298 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:49.368575 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:49.368852 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] PATH_INFO: `/v3/users/0b5e2ca5924c4741bab9dd4728bf14c8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:49.370025 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5e2ca5924c4741bab9dd4728bf14c8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:49.371985 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:49.418602 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5e2ca5924c4741bab9dd4728bf14c8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:49.419214 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5e2ca5924c4741bab9dd4728bf14c8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:49.426142 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5e2ca5924c4741bab9dd4728bf14c8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:49.438402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5e2ca5924c4741bab9dd4728bf14c8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:49.445404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5e2ca5924c4741bab9dd4728bf14c8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:49.452244 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1411/2835] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:11:49 2022] DELETE /identity/v3/users/0b5e2ca5924c4741bab9dd4728bf14c8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:49.457337 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bba79-c339-47d3-9b5e-0ab1b865aedc None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:49.457860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bba79-c339-47d3-9b5e-0ab1b865aedc None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:49.457972 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bba79-c339-47d3-9b5e-0ab1b865aedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:49.476809 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b72bba79-c339-47d3-9b5e-0ab1b865aedc None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:49.478234 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b72bba79-c339-47d3-9b5e-0ab1b865aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:49.536128 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1425/2836] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:49.546110 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:49.547558 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:49.603744 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b654b2f-8401-4fe0-bb6b-d005664689b6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:49.607327 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d4ae-0c10-4858-b151-3ae855cd7ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:49.607932 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d4ae-0c10-4858-b151-3ae855cd7ecf service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:49.608213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d4ae-0c10-4858-b151-3ae855cd7ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:49.608484 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d4ae-0c10-4858-b151-3ae855cd7ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:49.617331 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162d4ae-0c10-4858-b151-3ae855cd7ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:49.619318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162d4ae-0c10-4858-b151-3ae855cd7ecf service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:49.628635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1412/2837] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:49.899195 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bba79-c339-47d3-9b5e-0ab1b865aedc None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:49.900676 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b72bba79-c339-47d3-9b5e-0ab1b865aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:49.912905 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bba79-c339-47d3-9b5e-0ab1b865aedc None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:49.918916 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:49.919692 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:49.919984 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:49.920261 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] PATH_INFO: `/v3/projects/d1c0ddb6ad9b4dc39d7452c57b70faa4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:49.921434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1c0ddb6ad9b4dc39d7452c57b70faa4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:49.923404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:49.970449 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1c0ddb6ad9b4dc39d7452c57b70faa4', 'request_id': 'req-455fddd8-c284-4782-ad7e-95e9ae17662f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:49.976026 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1c0ddb6ad9b4dc39d7452c57b70faa4', 'request_id': 'req-455fddd8-c284-4782-ad7e-95e9ae17662f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:49.982903 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1c0ddb6ad9b4dc39d7452c57b70faa4', 'request_id': 'req-455fddd8-c284-4782-ad7e-95e9ae17662f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:49.986438 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:49.998121 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] The token cache is being invalidate because project d1c0ddb6ad9b4dc39d7452c57b70faa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:11:49.998404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:49.999083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:50.000606 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1426/2838] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:11:49 2022] DELETE /identity/v3/projects/d1c0ddb6ad9b4dc39d7452c57b70faa4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:50.004611 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d4ae-0c10-4858-b151-3ae855cd7ecf service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:50.014074 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d4ae-0c10-4858-b151-3ae855cd7ecf service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:50.054516 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:50.071568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a946c4a-559e-4c13-b717-94f15fc26a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:50.072213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a946c4a-559e-4c13-b717-94f15fc26a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:50.072502 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a946c4a-559e-4c13-b717-94f15fc26a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:50.072782 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a946c4a-559e-4c13-b717-94f15fc26a0a admin admin] PATH_INFO: `/v3/users/b7e4c01f931c46acbbdc1963987daebe` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:50.073950 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a946c4a-559e-4c13-b717-94f15fc26a0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e4c01f931c46acbbdc1963987daebe)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:50.075937 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a946c4a-559e-4c13-b717-94f15fc26a0a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:50.125404 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4a946c4a-559e-4c13-b717-94f15fc26a0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e4c01f931c46acbbdc1963987daebe'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:50.126059 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4a946c4a-559e-4c13-b717-94f15fc26a0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e4c01f931c46acbbdc1963987daebe'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:50.133319 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4a946c4a-559e-4c13-b717-94f15fc26a0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e4c01f931c46acbbdc1963987daebe'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:50.145141 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4a946c4a-559e-4c13-b717-94f15fc26a0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e4c01f931c46acbbdc1963987daebe'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:50.152527 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4a946c4a-559e-4c13-b717-94f15fc26a0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e4c01f931c46acbbdc1963987daebe'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:50.158695 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1413/2839] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:11:50 2022] DELETE /identity/v3/users/b7e4c01f931c46acbbdc1963987daebe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:50.432579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:50.434085 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:50.456874 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fddd8-c284-4782-ad7e-95e9ae17662f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:50.513795 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:50.514440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:50.514619 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:50.514949 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] PATH_INFO: `/v3/projects/1cba8350deac4616a68a40b5ea2ae740` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:50.516093 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cba8350deac4616a68a40b5ea2ae740)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:50.517988 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:50.565803 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cba8350deac4616a68a40b5ea2ae740', 'request_id': 'req-c233399e-8842-439b-9c28-b19129870ab6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:50.571536 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cba8350deac4616a68a40b5ea2ae740', 'request_id': 'req-c233399e-8842-439b-9c28-b19129870ab6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:50.577764 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cba8350deac4616a68a40b5ea2ae740', 'request_id': 'req-c233399e-8842-439b-9c28-b19129870ab6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:50.590299 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] The token cache is being invalidate because project 1cba8350deac4616a68a40b5ea2ae740 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:11:50.590583 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:50.591227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c233399e-8842-439b-9c28-b19129870ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:50.592726 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1427/2840] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:11:50 2022] DELETE /identity/v3/projects/1cba8350deac4616a68a40b5ea2ae740 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:50.597550 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c2d3c-7153-4b09-b128-1a84a694908c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:50.598014 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c2d3c-7153-4b09-b128-1a84a694908c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:50.598124 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c2d3c-7153-4b09-b128-1a84a694908c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:50.617566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-392c2d3c-7153-4b09-b128-1a84a694908c None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:50.619400 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-392c2d3c-7153-4b09-b128-1a84a694908c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:50.676312 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1414/2841] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:50.684657 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9be4c-34c7-472a-b21a-9bae1e20957e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:50.684952 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9be4c-34c7-472a-b21a-9bae1e20957e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:50.685230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9be4c-34c7-472a-b21a-9bae1e20957e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:50.704354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b8b9be4c-34c7-472a-b21a-9bae1e20957e None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:50.705637 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b8b9be4c-34c7-472a-b21a-9bae1e20957e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:50.713853 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1428/2842] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:50.718022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c2d3c-7153-4b09-b128-1a84a694908c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:50.728300 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c2d3c-7153-4b09-b128-1a84a694908c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:50.732798 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69220bf-0129-490e-bae1-740fc62c12a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:50.733402 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69220bf-0129-490e-bae1-740fc62c12a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:50.733680 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69220bf-0129-490e-bae1-740fc62c12a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:50.733951 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69220bf-0129-490e-bae1-740fc62c12a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:50.734635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69220bf-0129-490e-bae1-740fc62c12a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:11:50.734982 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69220bf-0129-490e-bae1-740fc62c12a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:50.736557 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69220bf-0129-490e-bae1-740fc62c12a8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:50.748515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1415/2843] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 04:11:50.752710 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9be4c-34c7-472a-b21a-9bae1e20957e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:50.761941 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9be4c-34c7-472a-b21a-9bae1e20957e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:50.766597 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06953a17-60e2-471b-b8fe-c3ee6402f11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:50.767147 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06953a17-60e2-471b-b8fe-c3ee6402f11c admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:50.767428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06953a17-60e2-471b-b8fe-c3ee6402f11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:50.767700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06953a17-60e2-471b-b8fe-c3ee6402f11c admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:50.768391 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06953a17-60e2-471b-b8fe-c3ee6402f11c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:50.770304 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06953a17-60e2-471b-b8fe-c3ee6402f11c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:50.800238 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1429/2844] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:11:50 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:50.804313 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69220bf-0129-490e-bae1-740fc62c12a8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:50.815134 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69220bf-0129-490e-bae1-740fc62c12a8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:50.861746 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:50.873781 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ed9b5-ba22-4d83-a8d6-93335aae825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:50.874397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ed9b5-ba22-4d83-a8d6-93335aae825f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:50.874681 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ed9b5-ba22-4d83-a8d6-93335aae825f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:50.874983 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ed9b5-ba22-4d83-a8d6-93335aae825f admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:50.875656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ed9b5-ba22-4d83-a8d6-93335aae825f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:50.877569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ed9b5-ba22-4d83-a8d6-93335aae825f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:50.898164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1416/2845] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:11:50 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:11:50.903166 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06953a17-60e2-471b-b8fe-c3ee6402f11c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:50.913472 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06953a17-60e2-471b-b8fe-c3ee6402f11c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:50.919111 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d6786-ba96-47a4-8a3c-b36bd7a9e1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:50.919710 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d6786-ba96-47a4-8a3c-b36bd7a9e1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:50.919990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d6786-ba96-47a4-8a3c-b36bd7a9e1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:50.920292 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d6786-ba96-47a4-8a3c-b36bd7a9e1f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:50.920979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d6786-ba96-47a4-8a3c-b36bd7a9e1f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:50.922557 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d6786-ba96-47a4-8a3c-b36bd7a9e1f0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:50.932786 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1430/2846] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:50.936414 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ed9b5-ba22-4d83-a8d6-93335aae825f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:50.946355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ed9b5-ba22-4d83-a8d6-93335aae825f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:50.950970 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed70c1a-af0e-46f8-b872-c22017b232f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:50.951567 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed70c1a-af0e-46f8-b872-c22017b232f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:50.951849 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed70c1a-af0e-46f8-b872-c22017b232f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:50.952125 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed70c1a-af0e-46f8-b872-c22017b232f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:50.952818 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed70c1a-af0e-46f8-b872-c22017b232f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:50.954398 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed70c1a-af0e-46f8-b872-c22017b232f0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:50.968223 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1417/2847] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:50.972462 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d6786-ba96-47a4-8a3c-b36bd7a9e1f0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:50.983442 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d6786-ba96-47a4-8a3c-b36bd7a9e1f0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:50.988209 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b4552-d67f-466d-960c-8a4d9e25cae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:50.988861 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b4552-d67f-466d-960c-8a4d9e25cae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:50.989144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b4552-d67f-466d-960c-8a4d9e25cae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:50.989418 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b4552-d67f-466d-960c-8a4d9e25cae5 admin admin] PATH_INFO: `/v3/projects/99af46bac3a444a48fd139a9d891eea0/users/f42dfd74bc8744d391fc217f44fffd6e/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:51.003586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b4552-d67f-466d-960c-8a4d9e25cae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99af46bac3a444a48fd139a9d891eea0, user_id=f42dfd74bc8744d391fc217f44fffd6e, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:51.006212 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b4552-d67f-466d-960c-8a4d9e25cae5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:51.018508 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1431/2848] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:50 2022] PUT /identity/v3/projects/99af46bac3a444a48fd139a9d891eea0/users/f42dfd74bc8744d391fc217f44fffd6e/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:11:51.029134 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0c11-9e76-412c-8926-03ceb592f905 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:51.029420 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0c11-9e76-412c-8926-03ceb592f905 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:51.029696 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0c11-9e76-412c-8926-03ceb592f905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:51.057107 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-871f0c11-9e76-412c-8926-03ceb592f905 None None] MFA Rules not processed for user `f42dfd74bc8744d391fc217f44fffd6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:51.058335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-871f0c11-9e76-412c-8926-03ceb592f905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:51.121151 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1418/2849] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:51.131647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b4552-d67f-466d-960c-8a4d9e25cae5 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:51.133419 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c04b4552-d67f-466d-960c-8a4d9e25cae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:51.160336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:51.192378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b4552-d67f-466d-960c-8a4d9e25cae5 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:51.196100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4567f99-5194-474c-8d71-b4c8e71e5010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:51.196729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4567f99-5194-474c-8d71-b4c8e71e5010 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:51.197015 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4567f99-5194-474c-8d71-b4c8e71e5010 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:51.197294 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4567f99-5194-474c-8d71-b4c8e71e5010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:51.205601 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4567f99-5194-474c-8d71-b4c8e71e5010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:51.207067 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4567f99-5194-474c-8d71-b4c8e71e5010 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:51.216336 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1432/2850] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:54.210958 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f0c11-9e76-412c-8926-03ceb592f905 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:54.273144 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f0c11-9e76-412c-8926-03ceb592f905 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:54.277816 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:54.278451 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:54.278737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:54.279044 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] PATH_INFO: `/v3/users/f42dfd74bc8744d391fc217f44fffd6e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:54.280213 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f42dfd74bc8744d391fc217f44fffd6e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:54.282120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:54.328087 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f42dfd74bc8744d391fc217f44fffd6e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:54.328739 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f42dfd74bc8744d391fc217f44fffd6e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:54.335760 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f42dfd74bc8744d391fc217f44fffd6e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:54.341279 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:54.349369 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f42dfd74bc8744d391fc217f44fffd6e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:54.357335 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f42dfd74bc8744d391fc217f44fffd6e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:54.363397 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1419/2851] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:11:54 2022] DELETE /identity/v3/users/f42dfd74bc8744d391fc217f44fffd6e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:54.368516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72436f41-ca56-4687-a47e-b71b00efb37f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:54.369122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72436f41-ca56-4687-a47e-b71b00efb37f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:54.369122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72436f41-ca56-4687-a47e-b71b00efb37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:54.387811 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-72436f41-ca56-4687-a47e-b71b00efb37f None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:54.389083 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-72436f41-ca56-4687-a47e-b71b00efb37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:54.448177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1433/2852] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:54.457608 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:54.459037 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:54.515643 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b68c65-2f70-41ba-aba2-f51a0ec5b6e7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:54.519528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8483bdf-27d2-4569-b5f7-fefb4000a827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:54.520143 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8483bdf-27d2-4569-b5f7-fefb4000a827 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:54.520422 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8483bdf-27d2-4569-b5f7-fefb4000a827 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:54.520694 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8483bdf-27d2-4569-b5f7-fefb4000a827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:54.529836 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8483bdf-27d2-4569-b5f7-fefb4000a827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:54.531296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8483bdf-27d2-4569-b5f7-fefb4000a827 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:54.541980 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1420/2853] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:11:54.814239 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72436f41-ca56-4687-a47e-b71b00efb37f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:54.815725 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-72436f41-ca56-4687-a47e-b71b00efb37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:54.829771 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72436f41-ca56-4687-a47e-b71b00efb37f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:54.834635 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:54.835792 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:54.836077 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:54.837042 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] PATH_INFO: `/v3/projects/99af46bac3a444a48fd139a9d891eea0` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:54.838325 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99af46bac3a444a48fd139a9d891eea0)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:54.840248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:54.887012 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99af46bac3a444a48fd139a9d891eea0', 'request_id': 'req-3ca42dd9-b62a-4b22-90f5-9294828713f3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:54.892406 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99af46bac3a444a48fd139a9d891eea0', 'request_id': 'req-3ca42dd9-b62a-4b22-90f5-9294828713f3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:54.898487 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99af46bac3a444a48fd139a9d891eea0', 'request_id': 'req-3ca42dd9-b62a-4b22-90f5-9294828713f3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:54.910833 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] The token cache is being invalidate because project 99af46bac3a444a48fd139a9d891eea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:11:54.911120 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:54.911733 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3ca42dd9-b62a-4b22-90f5-9294828713f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:11:54.913221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1434/2854] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:11:54 2022] DELETE /identity/v3/projects/99af46bac3a444a48fd139a9d891eea0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:11:54.919048 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281c86f-7ca3-4459-ad02-cbc0fc723ac5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:54.919342 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281c86f-7ca3-4459-ad02-cbc0fc723ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:54.919619 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281c86f-7ca3-4459-ad02-cbc0fc723ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:54.938530 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c281c86f-7ca3-4459-ad02-cbc0fc723ac5 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:54.939768 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c281c86f-7ca3-4459-ad02-cbc0fc723ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:54.997250 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1421/2855] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:55.004922 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d88c51-052c-438e-8ef9-d44e9587580a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:55.005214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d88c51-052c-438e-8ef9-d44e9587580a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:55.005496 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d88c51-052c-438e-8ef9-d44e9587580a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:55.026129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f8d88c51-052c-438e-8ef9-d44e9587580a None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:55.027396 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f8d88c51-052c-438e-8ef9-d44e9587580a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:55.035348 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1435/2856] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:55.040349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281c86f-7ca3-4459-ad02-cbc0fc723ac5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:55.049948 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281c86f-7ca3-4459-ad02-cbc0fc723ac5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:55.054421 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfcb93-504b-4de9-8920-5af1f70b0b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:55.055197 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfcb93-504b-4de9-8920-5af1f70b0b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:55.055521 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfcb93-504b-4de9-8920-5af1f70b0b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:55.055805 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfcb93-504b-4de9-8920-5af1f70b0b8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:55.056468 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfcb93-504b-4de9-8920-5af1f70b0b8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:11:55.056770 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfcb93-504b-4de9-8920-5af1f70b0b8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:55.058345 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfcb93-504b-4de9-8920-5af1f70b0b8a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:55.069462 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1422/2857] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:11:55.074242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d88c51-052c-438e-8ef9-d44e9587580a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:55.083426 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d88c51-052c-438e-8ef9-d44e9587580a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:55.088799 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9847e6-66df-47b2-a09d-0051010f6694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:55.089404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9847e6-66df-47b2-a09d-0051010f6694 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:55.089685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9847e6-66df-47b2-a09d-0051010f6694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:55.089961 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9847e6-66df-47b2-a09d-0051010f6694 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:55.090632 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9847e6-66df-47b2-a09d-0051010f6694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:55.092586 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9847e6-66df-47b2-a09d-0051010f6694 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:55.118648 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1436/2858] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:11:55 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:55.122837 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfcb93-504b-4de9-8920-5af1f70b0b8a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:55.132769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfcb93-504b-4de9-8920-5af1f70b0b8a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:55.187907 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88399769-4bb3-412f-8004-42ff5a1e910d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:55.188508 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88399769-4bb3-412f-8004-42ff5a1e910d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:55.188790 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88399769-4bb3-412f-8004-42ff5a1e910d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:55.189148 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88399769-4bb3-412f-8004-42ff5a1e910d admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:55.189721 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88399769-4bb3-412f-8004-42ff5a1e910d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:55.191697 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88399769-4bb3-412f-8004-42ff5a1e910d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:55.207326 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:55.214658 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1423/2859] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:11:55 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:11:55.218505 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9847e6-66df-47b2-a09d-0051010f6694 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:55.228729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9847e6-66df-47b2-a09d-0051010f6694 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:55.233322 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe0516-a117-4e18-b2af-45a00c818fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:55.233944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe0516-a117-4e18-b2af-45a00c818fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:55.234221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe0516-a117-4e18-b2af-45a00c818fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:55.234493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe0516-a117-4e18-b2af-45a00c818fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:55.235248 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe0516-a117-4e18-b2af-45a00c818fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:55.236829 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe0516-a117-4e18-b2af-45a00c818fc7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:55.246514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1437/2860] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:55.250368 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88399769-4bb3-412f-8004-42ff5a1e910d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:55.260728 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88399769-4bb3-412f-8004-42ff5a1e910d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:55.265373 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d5e1a-b3e8-4b12-ae34-660f0087c48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:55.265974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5e1a-b3e8-4b12-ae34-660f0087c48d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:55.266260 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5e1a-b3e8-4b12-ae34-660f0087c48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:55.266536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5e1a-b3e8-4b12-ae34-660f0087c48d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:55.267268 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d5e1a-b3e8-4b12-ae34-660f0087c48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:55.268851 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d5e1a-b3e8-4b12-ae34-660f0087c48d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:55.280021 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1424/2861] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:11:55.284039 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe0516-a117-4e18-b2af-45a00c818fc7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:55.294227 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe0516-a117-4e18-b2af-45a00c818fc7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:55.298737 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2f0af-9318-45be-aa8b-f89903808b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:55.299404 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2f0af-9318-45be-aa8b-f89903808b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:55.299685 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2f0af-9318-45be-aa8b-f89903808b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:55.299967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2f0af-9318-45be-aa8b-f89903808b99 admin admin] PATH_INFO: `/v3/projects/2fd80c87b0894dc1b9a92566c481e062/users/cccf0ddc27324dccb298f9ad075085d2/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:55.302973 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:11:55.316699 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2f0af-9318-45be-aa8b-f89903808b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd80c87b0894dc1b9a92566c481e062, user_id=cccf0ddc27324dccb298f9ad075085d2, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:55.319351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2f0af-9318-45be-aa8b-f89903808b99 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:55.332106 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1438/2862] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:11:55 2022] PUT /identity/v3/projects/2fd80c87b0894dc1b9a92566c481e062/users/cccf0ddc27324dccb298f9ad075085d2/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:11:55.342559 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2963ff-2a7e-41a5-92ae-0e8b117a012a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:55.342879 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2963ff-2a7e-41a5-92ae-0e8b117a012a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:55.343165 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2963ff-2a7e-41a5-92ae-0e8b117a012a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:55.371540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ac2963ff-2a7e-41a5-92ae-0e8b117a012a None None] MFA Rules not processed for user `cccf0ddc27324dccb298f9ad075085d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:11:55.372753 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ac2963ff-2a7e-41a5-92ae-0e8b117a012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:55.436918 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1425/2863] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:11:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:11:55.447747 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2f0af-9318-45be-aa8b-f89903808b99 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:11:55.449478 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bae2f0af-9318-45be-aa8b-f89903808b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:11:55.506490 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2f0af-9318-45be-aa8b-f89903808b99 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:11:55.510204 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c25f93-9684-4474-876e-fb9bc24a2208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2cf2558517445c79e47c3b67501ca09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:11:55.510849 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c25f93-9684-4474-876e-fb9bc24a2208 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:11:55.511129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c25f93-9684-4474-876e-fb9bc24a2208 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:11:55.511424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c25f93-9684-4474-876e-fb9bc24a2208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:11:55.519622 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c25f93-9684-4474-876e-fb9bc24a2208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:11:55.521038 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c25f93-9684-4474-876e-fb9bc24a2208 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:11:55.530384 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1439/2864] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:11:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:12:05.434053 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=38 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:12:05.526314 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:12:05.663777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2963ff-2a7e-41a5-92ae-0e8b117a012a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:05.727506 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2963ff-2a7e-41a5-92ae-0e8b117a012a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:05.731894 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:05.732528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:05.732808 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:05.733088 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] PATH_INFO: `/v3/users/cccf0ddc27324dccb298f9ad075085d2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:05.734239 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cccf0ddc27324dccb298f9ad075085d2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:05.736175 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:05.781881 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cccf0ddc27324dccb298f9ad075085d2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:12:05.782587 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cccf0ddc27324dccb298f9ad075085d2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:12:05.790802 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cccf0ddc27324dccb298f9ad075085d2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:12:05.803022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cccf0ddc27324dccb298f9ad075085d2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:12:05.810641 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cccf0ddc27324dccb298f9ad075085d2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:12:05.816568 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1426/2865] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:12:05 2022] DELETE /identity/v3/users/cccf0ddc27324dccb298f9ad075085d2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:12:05.821340 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c74c49-849a-41ed-9bbb-ed306e407693 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:05.821626 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c74c49-849a-41ed-9bbb-ed306e407693 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:05.821929 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c74c49-849a-41ed-9bbb-ed306e407693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:05.840364 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-98c74c49-849a-41ed-9bbb-ed306e407693 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:12:05.841560 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-98c74c49-849a-41ed-9bbb-ed306e407693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:05.901007 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1440/2866] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 04:12:05.910947 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:05.912349 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:05.968777 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bf4c0-cb83-45b4-bac5-cf02dcd9901b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:05.972426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b478323-2a64-4945-97aa-e2b19751f2db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:05.973034 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478323-2a64-4945-97aa-e2b19751f2db service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:05.973318 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478323-2a64-4945-97aa-e2b19751f2db service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:05.973596 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478323-2a64-4945-97aa-e2b19751f2db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:05.983106 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b478323-2a64-4945-97aa-e2b19751f2db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:05.984520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b478323-2a64-4945-97aa-e2b19751f2db service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:05.993989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1427/2867] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:12:06.256979 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c74c49-849a-41ed-9bbb-ed306e407693 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:06.259351 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-98c74c49-849a-41ed-9bbb-ed306e407693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:06.271474 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c74c49-849a-41ed-9bbb-ed306e407693 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:06.277036 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:06.277683 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:06.277966 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:06.278242 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] PATH_INFO: `/v3/projects/2fd80c87b0894dc1b9a92566c481e062` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:06.279434 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd80c87b0894dc1b9a92566c481e062)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:06.281361 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:06.328122 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd80c87b0894dc1b9a92566c481e062', 'request_id': 'req-39e5780d-51b9-46b0-be72-df70b663b23d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:12:06.333443 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd80c87b0894dc1b9a92566c481e062', 'request_id': 'req-39e5780d-51b9-46b0-be72-df70b663b23d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:12:06.339482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd80c87b0894dc1b9a92566c481e062', 'request_id': 'req-39e5780d-51b9-46b0-be72-df70b663b23d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:12:06.351460 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] The token cache is being invalidate because project 2fd80c87b0894dc1b9a92566c481e062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:12:06.351744 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:12:06.352908 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-39e5780d-51b9-46b0-be72-df70b663b23d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:12:06.354594 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1441/2868] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:12:06 2022] DELETE /identity/v3/projects/2fd80c87b0894dc1b9a92566c481e062 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:12:06.359525 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c194c8-5f6b-46b6-98b5-3eee42e96177 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:06.359819 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c194c8-5f6b-46b6-98b5-3eee42e96177 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:06.360100 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c194c8-5f6b-46b6-98b5-3eee42e96177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:06.380233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e6c194c8-5f6b-46b6-98b5-3eee42e96177 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:12:06.381450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e6c194c8-5f6b-46b6-98b5-3eee42e96177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:06.398522 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:12:06.444159 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1428/2869] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:12:06.453268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c009ad6-820d-446b-8879-7f9b1c9a2433 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:06.453558 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c009ad6-820d-446b-8879-7f9b1c9a2433 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:06.453862 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c009ad6-820d-446b-8879-7f9b1c9a2433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:06.473642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5c009ad6-820d-446b-8879-7f9b1c9a2433 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:12:06.474895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5c009ad6-820d-446b-8879-7f9b1c9a2433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:06.483326 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1442/2870] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:12:06.487560 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c194c8-5f6b-46b6-98b5-3eee42e96177 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:06.498215 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c194c8-5f6b-46b6-98b5-3eee42e96177 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:06.502960 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2552c4-d0df-480a-871e-9cabb3b9ed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:06.503578 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2552c4-d0df-480a-871e-9cabb3b9ed30 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:06.503863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2552c4-d0df-480a-871e-9cabb3b9ed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:06.504143 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2552c4-d0df-480a-871e-9cabb3b9ed30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:06.504846 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2552c4-d0df-480a-871e-9cabb3b9ed30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 04:12:06.505157 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2552c4-d0df-480a-871e-9cabb3b9ed30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:06.506744 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2552c4-d0df-480a-871e-9cabb3b9ed30 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:06.518569 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1429/2871] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Jul 28 04:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 04:12:06.522967 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c009ad6-820d-446b-8879-7f9b1c9a2433 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:06.532425 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c009ad6-820d-446b-8879-7f9b1c9a2433 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:06.537143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0c3c3-4af5-4f95-bbd3-596adc8d3b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:06.537776 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0c3c3-4af5-4f95-bbd3-596adc8d3b1e admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:06.538024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0c3c3-4af5-4f95-bbd3-596adc8d3b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:06.538297 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0c3c3-4af5-4f95-bbd3-596adc8d3b1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:06.538993 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0c3c3-4af5-4f95-bbd3-596adc8d3b1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:06.541892 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0c3c3-4af5-4f95-bbd3-596adc8d3b1e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:06.571140 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1443/2872] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:12:06 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:12:06.575247 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2552c4-d0df-480a-871e-9cabb3b9ed30 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:06.585258 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2552c4-d0df-480a-871e-9cabb3b9ed30 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:06.641588 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bd6be-35eb-41ce-8267-d69e8e273f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:06.642233 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bd6be-35eb-41ce-8267-d69e8e273f36 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:06.642520 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bd6be-35eb-41ce-8267-d69e8e273f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:06.642827 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bd6be-35eb-41ce-8267-d69e8e273f36 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:06.643518 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bd6be-35eb-41ce-8267-d69e8e273f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:06.645439 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bd6be-35eb-41ce-8267-d69e8e273f36 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:06.667231 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1430/2873] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:12:06 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:12:06.671428 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0c3c3-4af5-4f95-bbd3-596adc8d3b1e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:06.682075 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0c3c3-4af5-4f95-bbd3-596adc8d3b1e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:06.686729 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754e5bd-cc49-4952-9502-34d0f784a73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:06.687362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754e5bd-cc49-4952-9502-34d0f784a73c admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:06.687647 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754e5bd-cc49-4952-9502-34d0f784a73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:06.687923 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754e5bd-cc49-4952-9502-34d0f784a73c admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:06.688617 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754e5bd-cc49-4952-9502-34d0f784a73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:06.690229 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754e5bd-cc49-4952-9502-34d0f784a73c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:06.700271 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1444/2874] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:12:06.703981 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bd6be-35eb-41ce-8267-d69e8e273f36 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:06.714529 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bd6be-35eb-41ce-8267-d69e8e273f36 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:06.719120 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93157137-e38a-47df-b1ef-f2a5d2938a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:06.719714 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93157137-e38a-47df-b1ef-f2a5d2938a7a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:06.719994 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93157137-e38a-47df-b1ef-f2a5d2938a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:06.720296 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93157137-e38a-47df-b1ef-f2a5d2938a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:06.720976 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93157137-e38a-47df-b1ef-f2a5d2938a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:06.722554 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93157137-e38a-47df-b1ef-f2a5d2938a7a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:06.735931 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1431/2875] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:12:06.739879 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754e5bd-cc49-4952-9502-34d0f784a73c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:06.749982 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754e5bd-cc49-4952-9502-34d0f784a73c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:06.754584 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c96f0-e7a7-4205-97e8-05fc23f13f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:06.755268 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c96f0-e7a7-4205-97e8-05fc23f13f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:06.755554 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c96f0-e7a7-4205-97e8-05fc23f13f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:06.755856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c96f0-e7a7-4205-97e8-05fc23f13f7e admin admin] PATH_INFO: `/v3/projects/c5bbf3cb340045809e176d501f0ff27f/users/93681c22430d4b88bdb1d6bae885f73d/roles/f66cbbcc2491438184f04522c2a68578` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:06.770057 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c96f0-e7a7-4205-97e8-05fc23f13f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5bbf3cb340045809e176d501f0ff27f, user_id=93681c22430d4b88bdb1d6bae885f73d, role_id=f66cbbcc2491438184f04522c2a68578)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:06.772722 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c96f0-e7a7-4205-97e8-05fc23f13f7e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:06.786256 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1445/2876] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:12:06 2022] PUT /identity/v3/projects/c5bbf3cb340045809e176d501f0ff27f/users/93681c22430d4b88bdb1d6bae885f73d/roles/f66cbbcc2491438184f04522c2a68578 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:12:06.792168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5a5f6-6f0a-4477-a0f0-bfa2925df4b1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:06.792474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5a5f6-6f0a-4477-a0f0-bfa2925df4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:06.792744 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5a5f6-6f0a-4477-a0f0-bfa2925df4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:06.811276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f4e5a5f6-6f0a-4477-a0f0-bfa2925df4b1 None None] MFA Rules not processed for user `fe2947ccbc264fdba860153b59471d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:12:06.812474 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f4e5a5f6-6f0a-4477-a0f0-bfa2925df4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:06.869228 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1432/2877] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:12:06.878671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c96f0-e7a7-4205-97e8-05fc23f13f7e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:06.880384 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-026c96f0-e7a7-4205-97e8-05fc23f13f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:06.903021 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:12:06.941766 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c96f0-e7a7-4205-97e8-05fc23f13f7e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:06.945422 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf17f1c-4ca1-462a-9ab9-e3241a70655b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:06.946029 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf17f1c-4ca1-462a-9ab9-e3241a70655b service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:06.946307 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf17f1c-4ca1-462a-9ab9-e3241a70655b service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:06.946579 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf17f1c-4ca1-462a-9ab9-e3241a70655b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:06.955463 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf17f1c-4ca1-462a-9ab9-e3241a70655b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:06.956828 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf17f1c-4ca1-462a-9ab9-e3241a70655b service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:06.966479 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1446/2878] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:12:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:12:10.950438 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287be02-7cf7-481d-bcd5-9e8d87134be3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:10.951025 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287be02-7cf7-481d-bcd5-9e8d87134be3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:10.951025 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287be02-7cf7-481d-bcd5-9e8d87134be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:10.986931 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c287be02-7cf7-481d-bcd5-9e8d87134be3 None None] MFA Rules not processed for user `93681c22430d4b88bdb1d6bae885f73d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:12:10.988164 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c287be02-7cf7-481d-bcd5-9e8d87134be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:11.007620 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:12:11.060389 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1433/2879] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:12:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:12:11.064775 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf17f1c-4ca1-462a-9ab9-e3241a70655b service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:11.075471 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf17f1c-4ca1-462a-9ab9-e3241a70655b service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:11.080289 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd298bf-235b-4cde-876a-c244e13401b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:11.080895 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd298bf-235b-4cde-876a-c244e13401b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:11.081177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd298bf-235b-4cde-876a-c244e13401b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:11.081451 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd298bf-235b-4cde-876a-c244e13401b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:11.082124 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd298bf-235b-4cde-876a-c244e13401b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:11.084054 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd298bf-235b-4cde-876a-c244e13401b5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:11.116515 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1447/2880] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Jul 28 04:12:11 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 04:12:11.117533 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287be02-7cf7-481d-bcd5-9e8d87134be3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:11.128815 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287be02-7cf7-481d-bcd5-9e8d87134be3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:11.186737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb63051-45be-4252-9653-b39a47ed3ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:11.187303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63051-45be-4252-9653-b39a47ed3ffa admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:11.187613 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63051-45be-4252-9653-b39a47ed3ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:11.187890 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63051-45be-4252-9653-b39a47ed3ffa admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:11.188555 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb63051-45be-4252-9653-b39a47ed3ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:11.190467 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb63051-45be-4252-9653-b39a47ed3ffa admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:11.213138 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1434/2881] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jul 28 04:12:11 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 04:12:11.217064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd298bf-235b-4cde-876a-c244e13401b5 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:11.227525 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd298bf-235b-4cde-876a-c244e13401b5 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:11.232224 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a66bf-7950-4ca8-ad5f-6c0a6b9cf11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:11.232839 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a66bf-7950-4ca8-ad5f-6c0a6b9cf11b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:11.233125 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a66bf-7950-4ca8-ad5f-6c0a6b9cf11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:11.233403 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a66bf-7950-4ca8-ad5f-6c0a6b9cf11b admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:11.234101 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a66bf-7950-4ca8-ad5f-6c0a6b9cf11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:11.235960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a66bf-7950-4ca8-ad5f-6c0a6b9cf11b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:11.248856 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1448/2882] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:12:11.253002 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb63051-45be-4252-9653-b39a47ed3ffa admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:11.263922 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb63051-45be-4252-9653-b39a47ed3ffa admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:11.268543 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d01bf-803d-48fa-a54c-8d5eb1222c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:11.269206 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d01bf-803d-48fa-a54c-8d5eb1222c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:11.269493 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d01bf-803d-48fa-a54c-8d5eb1222c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:11.269776 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d01bf-803d-48fa-a54c-8d5eb1222c84 admin admin] PATH_INFO: `/v3/projects/733b2187f047419ca1012f657f724242/users/fce95f3eaf3248e9a80190aae0c7b550/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:11.285631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d01bf-803d-48fa-a54c-8d5eb1222c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=733b2187f047419ca1012f657f724242, user_id=fce95f3eaf3248e9a80190aae0c7b550, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:11.288328 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d01bf-803d-48fa-a54c-8d5eb1222c84 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:11.301407 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1435/2883] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:12:11 2022] PUT /identity/v3/projects/733b2187f047419ca1012f657f724242/users/fce95f3eaf3248e9a80190aae0c7b550/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 04:12:11.306493 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a66bf-7950-4ca8-ad5f-6c0a6b9cf11b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:11.316312 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a66bf-7950-4ca8-ad5f-6c0a6b9cf11b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:11.375168 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92730551-8d9b-4681-a209-916c373b223f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:11.375790 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92730551-8d9b-4681-a209-916c373b223f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:11.376160 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92730551-8d9b-4681-a209-916c373b223f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:11.376327 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92730551-8d9b-4681-a209-916c373b223f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:11.377018 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92730551-8d9b-4681-a209-916c373b223f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:11.378646 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92730551-8d9b-4681-a209-916c373b223f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:11.393183 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1449/2884] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Jul 28 04:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 04:12:11.396581 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d01bf-803d-48fa-a54c-8d5eb1222c84 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:11.408248 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d01bf-803d-48fa-a54c-8d5eb1222c84 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:11.412956 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db771d95-fd74-4845-bf69-deaf36482112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:11.413631 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db771d95-fd74-4845-bf69-deaf36482112 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:11.413919 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db771d95-fd74-4845-bf69-deaf36482112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:11.414196 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db771d95-fd74-4845-bf69-deaf36482112 admin admin] PATH_INFO: `/v3/projects/733b2187f047419ca1012f657f724242/users/fce95f3eaf3248e9a80190aae0c7b550/roles/a37ae24c516b48c69e13721c013134a2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:11.417078 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db771d95-fd74-4845-bf69-deaf36482112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=733b2187f047419ca1012f657f724242, user_id=fce95f3eaf3248e9a80190aae0c7b550, role_id=a37ae24c516b48c69e13721c013134a2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:11.419769 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db771d95-fd74-4845-bf69-deaf36482112 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:11.431465 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1436/2885] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Jul 28 04:12:11 2022] PUT /identity/v3/projects/733b2187f047419ca1012f657f724242/users/fce95f3eaf3248e9a80190aae0c7b550/roles/a37ae24c516b48c69e13721c013134a2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:12:15.862812 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d2892-ae5e-4af6-83e0-5a0679f54584 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:15.863356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d2892-ae5e-4af6-83e0-5a0679f54584 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:15.863356 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d2892-ae5e-4af6-83e0-5a0679f54584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:15.909358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-029d2892-ae5e-4af6-83e0-5a0679f54584 None None] MFA Rules not processed for user `fce95f3eaf3248e9a80190aae0c7b550`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:12:15.910430 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-029d2892-ae5e-4af6-83e0-5a0679f54584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:15.995440 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1450/2886] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:12:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:12:16.012303 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db771d95-fd74-4845-bf69-deaf36482112 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:16.015455 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-db771d95-fd74-4845-bf69-deaf36482112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:16.122200 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db771d95-fd74-4845-bf69-deaf36482112 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:16.127859 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac89ae-dac4-4c14-a302-d93e565c9be9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:16.128481 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac89ae-dac4-4c14-a302-d93e565c9be9 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:16.128771 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac89ae-dac4-4c14-a302-d93e565c9be9 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:16.129047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac89ae-dac4-4c14-a302-d93e565c9be9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:16.135352 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:12:16.189708 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac89ae-dac4-4c14-a302-d93e565c9be9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:16.191384 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac89ae-dac4-4c14-a302-d93e565c9be9 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:16.200977 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1437/2887] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:12:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:12:17.231100 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891bd8-ef7c-405a-a863-a4154442dc35 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:17.231590 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891bd8-ef7c-405a-a863-a4154442dc35 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:17.231696 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891bd8-ef7c-405a-a863-a4154442dc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:17.250576 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9d891bd8-ef7c-405a-a863-a4154442dc35 None None] MFA Rules not processed for user `93681c22430d4b88bdb1d6bae885f73d`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 04:12:17.251791 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9d891bd8-ef7c-405a-a863-a4154442dc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:17.259667 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1451/2888] 10.222.0.46 () {58 vars in 958 bytes} [Thu Jul 28 04:12:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 04:12:26.196817 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=24 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:12:27.245411 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:12:31.772840 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac89ae-dac4-4c14-a302-d93e565c9be9 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:31.774321 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bfac89ae-dac4-4c14-a302-d93e565c9be9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:31.830812 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac89ae-dac4-4c14-a302-d93e565c9be9 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:31.834418 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b968adb-0210-4171-b8a8-18a695f78ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:31.835062 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b968adb-0210-4171-b8a8-18a695f78ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:31.835350 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b968adb-0210-4171-b8a8-18a695f78ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:31.835628 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b968adb-0210-4171-b8a8-18a695f78ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:31.837382 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9b968adb-0210-4171-b8a8-18a695f78ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:12:31.846936 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b968adb-0210-4171-b8a8-18a695f78ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:31.848355 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b968adb-0210-4171-b8a8-18a695f78ab2 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:31.856779 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1438/2889] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:12:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:12:41.852647 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=26 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:12:46.715378 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891bd8-ef7c-405a-a863-a4154442dc35 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:12:46.725240 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891bd8-ef7c-405a-a863-a4154442dc35 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:12:46.728960 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d9564-2721-4591-a466-b4987d980e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:12:46.729568 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d9564-2721-4591-a466-b4987d980e71 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:12:46.729852 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d9564-2721-4591-a466-b4987d980e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:12:46.730129 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d9564-2721-4591-a466-b4987d980e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:12:46.740417 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d9564-2721-4591-a466-b4987d980e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:12:46.741860 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d9564-2721-4591-a466-b4987d980e71 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:12:46.752309 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1452/2890] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:12:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:12:56.747758 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:13:40.874366 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b968adb-0210-4171-b8a8-18a695f78ab2 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:13:40.875807 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9b968adb-0210-4171-b8a8-18a695f78ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:13:40.944610 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b968adb-0210-4171-b8a8-18a695f78ab2 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:13:40.949646 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91b687-21ca-4ed7-b0c7-dd9cdd99193f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:13:40.950254 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91b687-21ca-4ed7-b0c7-dd9cdd99193f service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:13:40.950536 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91b687-21ca-4ed7-b0c7-dd9cdd99193f service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:13:40.950833 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91b687-21ca-4ed7-b0c7-dd9cdd99193f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:13:40.952598 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8b91b687-21ca-4ed7-b0c7-dd9cdd99193f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:13:40.963765 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91b687-21ca-4ed7-b0c7-dd9cdd99193f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:13:40.965394 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91b687-21ca-4ed7-b0c7-dd9cdd99193f service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:13:40.975924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1439/2891] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:13:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:13:50.971276 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=32 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:14:43.911754 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d9564-2721-4591-a466-b4987d980e71 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:14:43.921433 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d9564-2721-4591-a466-b4987d980e71 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:14:43.925103 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424f9bb-3aa7-4bd1-b0d1-69abdcdc8cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:14:43.925700 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424f9bb-3aa7-4bd1-b0d1-69abdcdc8cac service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:14:43.925992 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424f9bb-3aa7-4bd1-b0d1-69abdcdc8cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:14:43.926267 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424f9bb-3aa7-4bd1-b0d1-69abdcdc8cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:14:43.928070 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3424f9bb-3aa7-4bd1-b0d1-69abdcdc8cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:14:43.942599 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424f9bb-3aa7-4bd1-b0d1-69abdcdc8cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:14:43.944221 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424f9bb-3aa7-4bd1-b0d1-69abdcdc8cac service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:14:43.955612 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1453/2892] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:14:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:14:53.950419 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:14:56.840800 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91b687-21ca-4ed7-b0c7-dd9cdd99193f service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:14:56.850338 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91b687-21ca-4ed7-b0c7-dd9cdd99193f service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:14:56.853964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13721dd9-6f1f-42a9-a176-28dc18ecf70c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:14:56.854566 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13721dd9-6f1f-42a9-a176-28dc18ecf70c service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:14:56.854882 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13721dd9-6f1f-42a9-a176-28dc18ecf70c service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:14:56.855154 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13721dd9-6f1f-42a9-a176-28dc18ecf70c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:14:56.856924 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-13721dd9-6f1f-42a9-a176-28dc18ecf70c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:14:56.870162 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13721dd9-6f1f-42a9-a176-28dc18ecf70c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:14:56.871964 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13721dd9-6f1f-42a9-a176-28dc18ecf70c service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:14:56.883316 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1440/2893] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:14:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:14:57.156365 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424f9bb-3aa7-4bd1-b0d1-69abdcdc8cac service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:14:57.165295 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424f9bb-3aa7-4bd1-b0d1-69abdcdc8cac service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:14:57.169223 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee416ffb-9ad8-4cc4-b5a5-61321a04bd71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:14:57.170393 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee416ffb-9ad8-4cc4-b5a5-61321a04bd71 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:14:57.170671 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee416ffb-9ad8-4cc4-b5a5-61321a04bd71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:14:57.170971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee416ffb-9ad8-4cc4-b5a5-61321a04bd71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:14:57.172834 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ee416ffb-9ad8-4cc4-b5a5-61321a04bd71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:14:57.183510 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee416ffb-9ad8-4cc4-b5a5-61321a04bd71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:14:57.184971 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee416ffb-9ad8-4cc4-b5a5-61321a04bd71 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:14:57.194534 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1454/2894] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:14:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:14:57.292827 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13721dd9-6f1f-42a9-a176-28dc18ecf70c service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:14:57.301916 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13721dd9-6f1f-42a9-a176-28dc18ecf70c service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:14:57.305552 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3158257-7ec0-4264-95a3-dc44546e6194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:14:57.306168 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3158257-7ec0-4264-95a3-dc44546e6194 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:14:57.306450 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3158257-7ec0-4264-95a3-dc44546e6194 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:14:57.306727 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3158257-7ec0-4264-95a3-dc44546e6194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:14:57.308540 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c3158257-7ec0-4264-95a3-dc44546e6194 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:14:57.322992 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3158257-7ec0-4264-95a3-dc44546e6194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:14:57.324426 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3158257-7ec0-4264-95a3-dc44546e6194 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:14:57.335440 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1441/2895] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:14:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:15:07.191035 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:15:07.330635 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=12 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:15:39.620215 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee416ffb-9ad8-4cc4-b5a5-61321a04bd71 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:15:39.629531 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee416ffb-9ad8-4cc4-b5a5-61321a04bd71 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:15:39.633177 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fe6b0-99de-415d-8bd2-58872df75a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e31408bfea4412d8f68aea9b3c7528d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:15:39.633784 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fe6b0-99de-415d-8bd2-58872df75a0a service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:15:39.634064 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fe6b0-99de-415d-8bd2-58872df75a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:15:39.634339 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fe6b0-99de-415d-8bd2-58872df75a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:15:39.636115 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c15fe6b0-99de-415d-8bd2-58872df75a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:15:39.646631 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fe6b0-99de-415d-8bd2-58872df75a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:15:39.648091 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fe6b0-99de-415d-8bd2-58872df75a0a service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:15:39.656424 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1455/2896] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:15:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:15:49.652358 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:16:22.970709 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3158257-7ec0-4264-95a3-dc44546e6194 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:16:22.983877 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3158257-7ec0-4264-95a3-dc44546e6194 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:16:22.990133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a749d17-9980-4b1e-8ace-9bd5acc81783 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:16:22.990763 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a749d17-9980-4b1e-8ace-9bd5acc81783 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:16:22.991084 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a749d17-9980-4b1e-8ace-9bd5acc81783 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:16:22.991362 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a749d17-9980-4b1e-8ace-9bd5acc81783 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:16:22.993133 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7a749d17-9980-4b1e-8ace-9bd5acc81783 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:16:23.005993 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a749d17-9980-4b1e-8ace-9bd5acc81783 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:16:23.007627 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a749d17-9980-4b1e-8ace-9bd5acc81783 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:16:23.019623 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1442/2897] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:16:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 04:16:25.447275 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fe6b0-99de-415d-8bd2-58872df75a0a service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:16:25.461024 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fe6b0-99de-415d-8bd2-58872df75a0a service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:16:25.467514 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6fa95-ec20-4fbe-b27e-355778c686df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '162ba16d643747c287bfe8a8e59df235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:16:25.468354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6fa95-ec20-4fbe-b27e-355778c686df service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:16:25.468642 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6fa95-ec20-4fbe-b27e-355778c686df service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:16:25.468919 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6fa95-ec20-4fbe-b27e-355778c686df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:16:25.470944 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-43c6fa95-ec20-4fbe-b27e-355778c686df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:16:25.486250 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6fa95-ec20-4fbe-b27e-355778c686df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:16:25.487990 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6fa95-ec20-4fbe-b27e-355778c686df service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:16:25.500354 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1456/2898] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:16:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:16:33.015187 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:16:33.104264 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a749d17-9980-4b1e-8ace-9bd5acc81783 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:16:33.169022 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a749d17-9980-4b1e-8ace-9bd5acc81783 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:16:33.173639 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:16:33.174272 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:16:33.174548 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:16:33.174861 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] PATH_INFO: `/v3/users/93681c22430d4b88bdb1d6bae885f73d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:16:33.176049 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] RBAC: Authorizing `identity:delete_user(user_id=93681c22430d4b88bdb1d6bae885f73d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:16:33.177944 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:16:33.223077 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93681c22430d4b88bdb1d6bae885f73d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.223737 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93681c22430d4b88bdb1d6bae885f73d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.231863 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93681c22430d4b88bdb1d6bae885f73d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.243974 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93681c22430d4b88bdb1d6bae885f73d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.251064 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93681c22430d4b88bdb1d6bae885f73d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.256940 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1443/2899] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:16:33 2022] DELETE /identity/v3/users/93681c22430d4b88bdb1d6bae885f73d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:16:33.430502 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6fa95-ec20-4fbe-b27e-355778c686df service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:16:33.441071 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6fa95-ec20-4fbe-b27e-355778c686df service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:16:33.498620 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:16:33.499298 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:16:33.499585 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:16:33.499870 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] PATH_INFO: `/v3/projects/c5bbf3cb340045809e176d501f0ff27f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:16:33.501034 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5bbf3cb340045809e176d501f0ff27f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:16:33.503041 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:16:33.553144 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5bbf3cb340045809e176d501f0ff27f', 'request_id': 'req-8eb15458-2439-4c44-b0b5-99faaabc7474'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.559405 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5bbf3cb340045809e176d501f0ff27f', 'request_id': 'req-8eb15458-2439-4c44-b0b5-99faaabc7474'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.566214 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5bbf3cb340045809e176d501f0ff27f', 'request_id': 'req-8eb15458-2439-4c44-b0b5-99faaabc7474'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.579230 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] The token cache is being invalidate because project c5bbf3cb340045809e176d501f0ff27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:16:33.579519 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.580143 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.581651 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1457/2900] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:16:33 2022] DELETE /identity/v3/projects/c5bbf3cb340045809e176d501f0ff27f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:16:33.585691 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:16:33.587226 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:16:33.600731 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc374ec-526b-44e9-865e-b5246c2dfacb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:16:33.659656 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:16:33.660299 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:16:33.660586 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:16:33.660865 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] PATH_INFO: `/v3/users/fce95f3eaf3248e9a80190aae0c7b550` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:16:33.662047 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce95f3eaf3248e9a80190aae0c7b550)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:16:33.663989 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:16:33.686336 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:16:33.711433 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce95f3eaf3248e9a80190aae0c7b550'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.712093 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce95f3eaf3248e9a80190aae0c7b550'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.720036 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce95f3eaf3248e9a80190aae0c7b550'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.732337 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce95f3eaf3248e9a80190aae0c7b550'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.739103 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce95f3eaf3248e9a80190aae0c7b550'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:33.745009 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1444/2901] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Jul 28 04:16:33 2022] DELETE /identity/v3/users/fce95f3eaf3248e9a80190aae0c7b550 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:16:33.929362 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:16:33.930995 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:16:33.945743 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb15458-2439-4c44-b0b5-99faaabc7474 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:16:34.004235 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe2947ccbc264fdba860153b59471d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826fca2fab6740f6870c6426848e8d90', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:16:34.004877 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:16:34.005163 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:16:34.005444 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] PATH_INFO: `/v3/projects/733b2187f047419ca1012f657f724242` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:16:34.006610 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] RBAC: Authorizing `identity:delete_project(project_id=733b2187f047419ca1012f657f724242)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:16:34.008547 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:16:34.031401 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:16:34.057772 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '733b2187f047419ca1012f657f724242', 'request_id': 'req-0133ada6-0d70-4974-ad99-591cff89b07f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:34.063516 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '733b2187f047419ca1012f657f724242', 'request_id': 'req-0133ada6-0d70-4974-ad99-591cff89b07f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:34.069482 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '733b2187f047419ca1012f657f724242', 'request_id': 'req-0133ada6-0d70-4974-ad99-591cff89b07f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:34.082127 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] The token cache is being invalidate because project 733b2187f047419ca1012f657f724242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 04:16:34.082407 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:34.083048 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0133ada6-0d70-4974-ad99-591cff89b07f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 04:16:34.084561 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1458/2902] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Jul 28 04:16:33 2022] DELETE /identity/v3/projects/733b2187f047419ca1012f657f724242 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 04:16:43.741534 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:16:44.079155 dv-cinder-799871-47-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 04:18:46.865941 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 04:18:46.867413 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:18:47.006277 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaabf3f-45e4-45e3-b930-1724af5397c6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 04:18:47.062589 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f601db-864a-4aa5-877a-c330ebe3da88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '551281a31ce24e2f9b81b36f69419311', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7d94dcd9ec4eea9c5385b3050ff5df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 04:18:47.063237 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f601db-864a-4aa5-877a-c330ebe3da88 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 04:18:47.063515 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f601db-864a-4aa5-877a-c330ebe3da88 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 04:18:47.063790 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f601db-864a-4aa5-877a-c330ebe3da88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 04:18:47.065528 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-24f601db-864a-4aa5-877a-c330ebe3da88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 04:18:47.077153 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f601db-864a-4aa5-877a-c330ebe3da88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 04:18:47.078716 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f601db-864a-4aa5-877a-c330ebe3da88 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 04:18:47.090673 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1445/2903] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jul 28 04:18:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 04:18:57.086535 dv-cinder-799871-47-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}